Backtesting Without Fooling Yourself¶
Lookahead bias, overfitting and real transaction costs — demonstrated, not just warned about.
Part 29 of 36 in the ServLoci algo/options trading notebook series — full index in notebooks/README.md.
Setup — no broker account needed¶
!pip install -q yfinance
import yfinance as yf
import pandas as pd
import numpy as np
# No broker account, no API key, no ServLoci setup needed for this chapter —
# yfinance reads public end-of-day data from Yahoo Finance. Indian tickers take
# an ".NS" suffix (NSE) or ".BO" (BSE); index tickers are prefixed with "^"
# (^NSEI = Nifty 50, ^BSESN = Sensex, ^GSPC = S&P 500).
NSE_TICKER = "RELIANCE.NS"
US_TICKER = "AAPL"
INDEX_TICKER = "^NSEI"
print("yfinance", yf.__version__)
Notebook 14 said it plainly: "a simplified, cost-free backtest"¶
Notebook 14 backtests a weekly short straddle and is honest about its own limits in its first cell: no slippage, no costs, no margin, no out-of-sample split. That's a reasonable thing for a mechanics demo to skip — but it means notebook 14's P&L number is not evidence of an edge, and most retail backtesting content (free courses especially) stops exactly there, mentioning "backtesting pitfalls" in a bullet list without ever showing you the bug in running code.
This notebook does the opposite: it deliberately writes two classic backtesting mistakes, runs them, and then fixes them — so you can see the difference a subtle bug makes to a P&L curve, not just be told it matters. Then it adds back the two things notebook 14 skipped that matter most in India: transaction costs and an out-of-sample split.
hist = yf.download(NSE_TICKER, period="2y", interval="1d", progress=False, multi_level_index=False)
hist = hist[["Close"]].dropna()
hist.columns = ["close"]
print(f"{NSE_TICKER}: {len(hist)} sessions, {hist.index[0].date()} to {hist.index[-1].date()}")
Bug #1: look-ahead bias — using tomorrow's information today¶
The strategy: go long when a fast moving average crosses above a slow moving average ("golden cross"), flat otherwise. The buggy version below computes the signal, then aligns it to today's return using shift(-1) on the return series — which quietly hands the strategy tomorrow's closing price before the trading day it's supposed to act on has happened. This is an easy bug to write by accident (off-by-one direction errors in shift() are the single most common source of look-ahead bias in pandas backtests) and it always makes the backtest look better than any strategy could actually trade, because the "signal" is partly informed by the outcome it's predicting.
fast, slow = 20, 50
hist["sma_fast"] = hist["close"].rolling(fast).mean()
hist["sma_slow"] = hist["close"].rolling(slow).mean()
hist["signal"] = (hist["sma_fast"] > hist["sma_slow"]).astype(int)
hist["daily_ret"] = hist["close"].pct_change()
# BUGGY: shift(-1) pulls tomorrow's return back onto today's signal —
# the strategy is effectively told the outcome before it "happens."
buggy_strategy_ret = hist["signal"] * hist["daily_ret"].shift(-1)
buggy_equity = (1 + buggy_strategy_ret.fillna(0)).cumprod()
print("Look-ahead-biased final equity multiple:", round(buggy_equity.iloc[-1], 3))
The fix: only trade on information that existed at the time¶
The correct version uses today's signal to decide whether you're positioned for tomorrow's move, which means the signal itself must be built from data available before the bar it trades — shift the signal forward by one bar, not the return backward. The visible difference between the buggy and corrected equity curves is the entire size of the look-ahead bug — on real data, it is rarely small.
# CORRECT: shift the *signal* forward — trade tomorrow using only
# information known at today's close, never information from tomorrow.
hist["signal_lagged"] = hist["signal"].shift(1)
correct_strategy_ret = hist["signal_lagged"] * hist["daily_ret"]
correct_equity = (1 + correct_strategy_ret.fillna(0)).cumprod()
print("Look-ahead-corrected final equity multiple:", round(correct_equity.iloc[-1], 3))
print("Buy-and-hold final equity multiple: ", round((1 + hist['daily_ret'].fillna(0)).cumprod().iloc[-1], 3))
Bug #2: overfitting — picking the window that happened to win¶
A moving-average crossover has a free parameter: the window lengths. Grid-searching many window pairs against a fixed historical period and reporting the single best result as "the strategy's edge" is curve-fitting, not strategy design — you are, by construction, selecting for whichever parameter combination happened to fit this specific noise, which is not the same thing as a persistent, tradeable pattern.
results = []
for fast_w in range(5, 41, 5):
for slow_w in range(50, 121, 10):
if fast_w >= slow_w:
continue
sig = (hist["close"].rolling(fast_w).mean() > hist["close"].rolling(slow_w).mean()).astype(int)
ret = sig.shift(1) * hist["daily_ret"]
equity_mult = (1 + ret.fillna(0)).cumprod().iloc[-1]
results.append({"fast": fast_w, "slow": slow_w, "equity_mult": equity_mult})
grid = pd.DataFrame(results).sort_values("equity_mult", ascending=False)
print("Best in-sample combination (DO NOT trust this number yet):")
print(grid.head(3))
The fix: a walk-forward split — validate on data the search never saw¶
Split the history into a train window (search for the best parameters here) and a held-out test window (evaluate — do not re-tune — on this). If the parameter combination that won on the train window doesn't hold up on the untouched test window, the "edge" the grid search found was fit to noise in the train period, not a real, persistent pattern. This is the minimum bar for treating a backtest result as evidence rather than a coincidence; a real research process goes further with multiple rolling train/test folds, but even a single honest split catches the most common failure mode.
split_idx = int(len(hist) * 0.7)
train, test = hist.iloc[:split_idx], hist.iloc[split_idx:]
def equity_mult_for(frame, fast_w, slow_w):
sig = (frame["close"].rolling(fast_w).mean() > frame["close"].rolling(slow_w).mean()).astype(int)
ret = sig.shift(1) * frame["close"].pct_change()
return float((1 + ret.fillna(0)).cumprod().iloc[-1])
best_fast, best_slow = int(grid.iloc[0]["fast"]), int(grid.iloc[0]["slow"])
train_mult = equity_mult_for(train, best_fast, best_slow)
test_mult = equity_mult_for(test, best_fast, best_slow)
print(f"Best train-window combo ({best_fast}/{best_slow}) — train equity mult: {train_mult:.3f}")
print(f"Same combo, held-out test window — test equity mult: {test_mult:.3f}")
print("A test result that's dramatically worse than train is the walk-forward split doing its job.")
What notebook 14 left out: real transaction costs¶
India-specific costs that a gross P&L number ignores entirely: STT (securities transaction tax — charged differently for equity delivery, intraday and F&O, and rates change by government notification, so verify the current rate on the NSE/SEBI circular before using this for real sizing, don't trust a number in a training notebook), brokerage per executed order, exchange transaction charges, and slippage — the gap between the price your signal used and the price you actually got filled at, which is usually small for a large-cap NSE stock and can be large for anything illiquid.
The mechanism below is a flat cost-per-trade assumption, deliberately conservative and easy to swap for real numbers from your own broker's contract note.
# Illustrative flat-cost assumption — replace with your broker's actual
# brokerage + STT + exchange charges before drawing any real conclusion.
cost_per_trade_pct = 0.001 # 0.10% round-trip, placeholder — verify real STT/brokerage rates
trades = hist["signal_lagged"].diff().fillna(0) != 0 # a trade happens whenever the position changes
n_trades = int(trades.sum())
gross_mult = correct_equity.iloc[-1]
cost_drag = n_trades * cost_per_trade_pct
net_mult = gross_mult * (1 - cost_drag)
print(f"Trades over the period: {n_trades}")
print(f"Gross equity multiple (no costs): {gross_mult:.3f}")
print(f"Net equity multiple (with costs): {net_mult:.3f}")
print("A strategy that trades often can look profitable gross and lose money net — "
"the more frequently a signal flips, the more this gap matters.")
Putting it together¶
Every fix in this notebook — lagging the signal correctly, validating on a held-out window, and subtracting real costs — makes a backtest less impressive and more trustworthy. That trade is the entire point: notebook 14's cost-free, single-window straddle backtest is a fine way to check that payoff mechanics are wired correctly, but treat any backtest result, including the ones in this series, as a hypothesis to keep stress-testing, not a number to size real capital against.
« Previous: Options, Priced Against Reality
Next: Position Sizing, Risk of Ruin & Trading Psychology »
Try the concepts above interactively: Options Strategy Builder · Docs · Get your static IP