Skip to main content

Backtesting Best Practices - Avoid These 5 Common Mistakes

· 4 min read
VecAlpha Team
VecAlpha Team
Quantitative Trading Platform

Backtesting is the foundation of quantitative trading. A robust backtest gives you confidence that your strategy will perform well in live markets. But a flawed backtest? It can lead to costly losses.

Why Backtesting Matters​

Before risking real capital, you need to answer:

  • Would this strategy have been profitable historically?
  • What's the maximum loss I could have experienced?
  • How does the strategy perform in different market conditions?

A well-designed backtest answers these questions. A poorly designed one gives false confidence.

Mistake #1: Ignoring Transaction Costs​

The Problem​

Many beginners run backtests with zero transaction costs. This dramatically overstates returns.

Consider a high-frequency strategy that trades 100 times per day. Even a 0.1% commission means:

Daily commission cost = 100 trades × 0.1% × $10,000 = $100
Annual cost = $100 × 252 trading days = $25,200

That's a 25% drag on a $100,000 account!

The Solution​

Always include realistic costs:

# VecAlpha backtest configuration
config = {
    'commission': 0.001,      # 0.1% per trade
    'slippage': 0.0005,       # 0.05% slippage
    'spread_cost': 0.0001,    # Bid-ask spread
}

Pro tip: Different markets have different cost structures. Crypto tends to have higher slippage than equities.

Mistake #2: Look-Ahead Bias​

The Problem​

Using information that wasn't available at the time of the trade. Examples:

  • Using the day's closing price to make an intraday decision
  • Calculating indicators with future data due to windowing errors
  • Training ML models on the full dataset before splitting

The Solution​

Be strict about data ordering:

# WRONG: Uses future data
df['signal'] = df['close'].shift(-1) > df['close']  # Tomorrow's close!

# CORRECT: Uses only past data
df['signal'] = df['close'].shift(1) > df['close'].shift(2)  # Yesterday's comparison

Test: Ask yourself - "Could I have known this at the exact moment of the trade?"

Mistake #3: Survivorship Bias​

The Problem​

Testing only on currently traded securities ignores companies that went bankrupt or were delisted.

Imagine backtesting a "buy and hold" strategy on tech stocks from 2010-2020. You'd pick Apple, Amazon, Google... but miss the hundreds of failed startups.

The Solution​

Use survivorship-free datasets that include:

  • Delisted stocks
  • Merged/acquired companies
  • Changed ticker symbols

VecAlpha provides survivorship-free data for all supported markets.

Mistake #4: Overfitting​

The Problem​

Optimizing parameters until the backtest looks perfect. This fits the strategy to historical noise, not real patterns.

Signs of overfitting:

  • Unrealistic returns (> 100% annually with low volatility)
  • Too many parameters
  • Sharp performance drop when testing on new data

The Solution​

Walk-forward analysis:

  1. Split data into chunks (e.g., 10 periods)
  2. For each period:
    • Optimize on previous periods
    • Test on current period (out-of-sample)
  3. Combine results from all out-of-sample periods
# VecAlpha walk-forward test
from vecalpha import WalkForwardAnalysis

wfa = WalkForwardAnalysis(
    train_period='2Y',      # 2 years for optimization
    test_period='6M',       # 6 months out-of-sample
    n_splits=5              # 5 walk-forward periods
)

results = wfa.run(strategy, data)
print(f"Out-of-sample Sharpe: {results.sharpe_ratio}")

Rule of thumb: If out-of-sample performance is less than 50% of in-sample, your strategy is overfitted.

Mistake #5: Ignoring Market Regime​

The Problem​

Strategies often work well in certain market conditions but fail in others:

  • A trend-following strategy thrives in trending markets but loses in sideways markets
  • A mean-reversion strategy does well in range-bound markets but gets crushed in trends

The Solution​

Regime-aware testing:

# Identify market regimes
def classify_regime(returns, window=60):
    """
    Classify market as Trending or Mean-Reverting
    based on the Hurst exponent.
    """
    # ... regime classification logic ...
    return regime

# Test performance by regime
for regime in ['trending', 'mean_reverting', 'volatile']:
    regime_returns = backtest(strategy, data, regime=regime)
    print(f"{regime}: Sharpe = {regime_returns.sharpe}")

Quick Checklist​

Before trusting a backtest, verify:

  • Realistic transaction costs included
  • No look-ahead bias in signals
  • Survivorship-free data used
  • Out-of-sample testing performed
  • Performance analyzed across market regimes
  • Position sizing and risk management applied

Next Steps​

Ready to run your own robust backtests?

What backtesting challenges have you faced? Share your experiences in the comments!