Seven backtesting pitfalls to avoid

A backtest answers a narrow question: “If these exact rules had been followed on this historical data, under these assumptions, what would have happened?” It cannot tell you what will happen next. Yet even that narrow question is easy to answer wrongly. Small modeling shortcuts compound, and the result is an equity curve that looks far smoother than anything you could have traded.

Below are seven pitfalls that show up again and again in strategy research, along with practical ways to detect them before they mislead you.

1. Look-ahead bias

Look-ahead bias happens when a simulated decision uses information that was not available at the moment the decision was made. The classic example is computing a signal from today's closing price and then assuming you traded at that same close. Subtler versions include using financial statements as of the fiscal period end rather than the date they were actually published, or normalizing a feature with the mean and standard deviation of the entire dataset.

The fix is to be explicit about timing. Decide when each signal becomes known and when the resulting order can realistically execute, then shift the signal accordingly.

import pandas as pd

df = pd.read_csv("prices.csv", parse_dates=["date"], index_col="date")
df["ret"] = df["close"].pct_change()
df["signal"] = (df["close"] > df["close"].rolling(50).mean()).astype(int)

# Wrong: today's signal earns today's return (it used today's close)
df["biased"] = df["signal"] * df["ret"]

# Better: the position is taken after the signal is known, so it earns the next return
df["strategy"] = df["signal"].shift(1) * df["ret"]

2. Survivorship bias

If your universe is built from the companies that exist today, you have quietly removed every firm that was delisted, acquired or went bankrupt during the test period. Those names often had poor returns before they disappeared, so excluding them flatters almost any long-only strategy. Use point-in-time constituent lists and a dataset that retains delisted securities, or at least be aware of the direction of the bias when you cannot.

3. Ignoring costs and slippage

Commissions, bid-ask spreads, exchange fees, borrowing costs for short positions and market impact all reduce returns. A strategy that trades frequently can look excellent before costs and unremarkable after them. Model costs per trade, measure turnover, and test how results change when you double your cost assumption. If a modest increase erases the edge, the edge was probably not robust.

4. Overfitting and data snooping

Every time you tweak a parameter after looking at the results, you use up some of the information in your data. Test enough combinations and one of them will look impressive purely by chance. Warning signs include many tunable parameters, rules with oddly specific thresholds, and performance that collapses when a parameter moves slightly.

• Keep a log of every variant you test, not just the winner.
• Reserve an out-of-sample period you do not look at until the design is frozen.
• Prefer parameter regions that perform reasonably well over a single sharp peak.

5. Unrealistic fills and liquidity

Backtests often assume that every order fills completely at the price you wanted. In practice, limit orders may not fill at all, market orders pay the spread, and large orders relative to traded volume move the price. Cap position sizes as a fraction of average volume, treat limit fills conservatively, and be skeptical of results that depend on trading thinly traded instruments.

6. Dirty or misaligned data

Unadjusted stock splits create fake crashes. Mixed time zones shift bars by hours. Missing sessions get forward-filled into flat prices that look like low volatility. Before trusting any result, check the data itself: plot it, scan for extreme one-bar returns, and confirm how dividends and corporate actions are handled. Many strategy “discoveries” turn out to be data errors.

7. Judging on too little history or the wrong metrics

A strategy tested over a single market regime may simply be a bet on that regime continuing. Test across different volatility and interest-rate environments where possible, and look beyond total return. Maximum drawdown, the length of drawdowns, the distribution of returns and the number of independent trades all matter. A result built on a handful of trades is an anecdote, not evidence.

A quick pre-flight checklist

1. Is every signal shifted so that it only uses information available at decision time?
2. Does the universe include securities that no longer exist?
3. Are costs, slippage and borrowing included, and have you stress-tested them?
4. Was an out-of-sample period kept untouched until the end?
5. Are fills and position sizes realistic for the instrument's liquidity?
6. Has the raw data been inspected for splits, gaps and time-zone issues?
7. Do the results hold across regimes and on risk metrics, not just returns?

A backtest that survives these checks is not a guarantee of anything. It is simply a result you can reason about honestly.

Want to go deeper? See how Quantzivo Studio's backtesting workflow is being designed around these checks, or build the underlying skills in the Factor Models & Quantitative Research course.