ETF Backtesting in Python: Compare a Rule with Buy and Hold

The popularity of ETF backtesting: Systematic beta managementCopy Anchor Link

In systematic quantitative finance, individual stock selection represents only a fraction of deployable strategies. While single-equity trading requires navigating idiosyncratic corporate announcements, earnings surprises, regulatory investigations, and bankruptcy risks, Exchange-Traded Funds (ETFs) aggregate dozens, hundreds, or thousands of underlying assets into a single liquid, exchange-traded vehicle.

For quantitative researchers, family offices, and systematic asset allocators, broad-market index ETFs—such as the SPDR S&P 500 ETF Trust (SPY), the Invesco QQQ Trust (QQQ), and the iShares Russell 2000 ETF (IWM)—are the premier instruments for systematic beta management. Rather than hunting for micro-cap alpha or forecasting company fundamentals, ETF quantitative models focus on macro regime detection, trend-following filters, and tail-risk mitigation.

An ETF backtest evaluating a systematic trend filter answers a fundamental institutional question: Can a deterministic rule improve risk-adjusted capital compounding relative to passive buy-and-hold?

┌────────────────────────────────────────────────────────────────────────┐
│                        Market Regime Spectrum                          │
├──────────────────────────────────┬─────────────────────────────────────┤
│  Bull Regime (Price > Trend)     │  Bear / Crash Regime (Price < Trend)│
│  ────────────────────────────     │  ───────────────────────────────────│
│  • Broad market expansion        │  • Volatility clustering & cascades │
│  • Low realized dispersion       │  • Severe drawdowns (-20% to -55%)  │
│  • Strategy Action: 100% Long    │  • Strategy Action: 100% Cash Buffer│
│  • Objective: Capture beta       │  • Objective: Capital preservation  │
└──────────────────────────────────┴─────────────────────────────────────┘

Because broad-market index ETFs cannot go bankrupt (an index mechanically drops insolvent constituents and replaces them with viable enterprises), they exhibit persistent long-term drift coupled with severe, multi-year cyclical drawdowns. Applying a quantitative rule—such as the classic 200-day Simple Moving Average (SMA) filter—aims to capture the vast majority of cyclical expansions while systematically de-risking into cash during prolonged secular bear markets.

However, designing an empirical ETF backtest requires accounting for ETF-specific market microstructure, tracking frictions, corporate action adjustments, and benchmark construction standards.

Unique considerations for ETF dataCopy Anchor Link

Although ETFs trade on secondary exchanges with the same mechanics as common equities, their underlying fund structure introduces distinct data and accounting nuances that directly impact backtest validity.

Institutional ETF Structure:
Underlying Index Basket ───> Daily NAV Calculation (Minus Expense Ratio Drag)
                                      │
                                      ▼
Secondary Exchange Trading ─> Market Price (Bid-Ask Spread, Premiums/Discounts)

1. Inception Dates and Historical DepthCopy Anchor Link

While bellwether index ETFs possess deep historical records (SPY launched in January 1993, QQQ in March 1999), many sector, thematic, and factor ETFs launched within the last 5 to 10 years.

Backtesting a trend-following rule exclusively on a fund with an inception date of 2017 introduces extreme recency bias. A test window spanning only 2017 to 2026 captures an aggressive central-bank liquidity expansion and a brief pandemic shock, but completely omits the prolonged, structural drawdowns of the 2000–2002 tech bubble burst or the 2007–2009 Global Financial Crisis. To evaluate tail-risk mitigation, your historical sample must span multiple complete macroeconomic market cycles.

2. Expense Ratios Deducted from Net Asset Value (NAV)Copy Anchor Link

Unlike hedge funds or managed accounts that invoice management fees periodically, an ETF's expense ratio is deducted daily and continuously from its Net Asset Value (NAV) prior to the calculation of official closing prices:

Daily NAV Adjustment = Gross NAV * (1 - Expense_Ratio / 252)

For ultra-liquid index funds like Vanguard S&P 500 ETF (VOO, 0.03% expense ratio) or SPY (0.09%), this daily drag is nominal. However, for specialized sector funds (0.45% to 0.75%) or actively managed ETFs (0.75% to 0.95%), the cumulative drag over a 15-year backtest is substantial. Because historical OHLCV data reflects prices net of expense ratios, passive buy-and-hold implicitly absorbs this drag. When your strategy rotates into cash, your capital ceases paying the fund's expense ratio, creating an asymmetric cost dynamic that must be understood during performance attribution.

3. Dividend Adjustments: Total Return vs. Price ReturnCopy Anchor Link

Broad-market ETFs distribute dividends quarterly. In quantitative research, relying on unadjusted closing prices severely distorts performance, as dividend distributions cause artificial price drops on ex-dividend dates:

P_{\text{ex}} = P_{\text{cum}} - \text{Dividend}

[!IMPORTANT] Always conduct long-horizon ETF backtests using Total Return (Dividend-Adjusted) data. Over a 20-year horizon on the S&P 500, dividend reinvestment accounts for more than 40% of cumulative portfolio equity.

4. Liquidity Profiles, Tracking Errors, and SpreadsCopy Anchor Link

Mega-cap ETFs (SPY, IVV, QQQ) feature sub-penny spreads, extreme institutional depth, and minimal tracking error relative to their benchmarks. In contrast, international, fixed-income, or commodity ETFs often exhibit noticeable tracking error and wider bid-ask spreads (5 to 25 basis points). Modeling zero transaction costs or zero slippage on non-mega-cap ETFs will produce heavily overstated strategy performance.

Constructing the benchmark: Normalized Buy-and-HoldCopy Anchor Link

Evaluating an active ETF strategy requires comparing its performance against an unleveraged Buy-and-Hold benchmark. However, a naive comparison frequently introduces subtle biases that distort performance metrics.

Strategy Execution Timeline:
Bar 0 ─────── Warm-Up (200 bars) ───────> Bar 200 (First Signal) ──────> Terminal Bar
                                                   │
Normalized Benchmark Timeline:                     ▼
Bar 0 ─── Cash Retention ────────────────> Bar 200 (Execute Buy & Hold) ─> Terminal Bar

The Rules of Honest Benchmark AlignmentCopy Anchor Link

1. Synchronized Execution Date: If your strategy utilizes a 200-day moving average, it cannot emit an informed trading signal until bar 200. Comparing a strategy that sits in uninvested cash for 200 bars against a benchmark that purchased at bar 0 distorts the initial capital trajectory. Both the active strategy and the benchmark should have their comparative capital paths evaluated from the first tradable bar.
2. Identical Starting Capital: The benchmark portfolio must be initialized with the exact initial cash balance (C_0 =100,000$) allocated to the strategy.
3. Symmetric Frictional Costs: The benchmark's initial position must incur the same transaction commission and opening slippage applied to strategy trades.
4. Mark-to-Market Consistency: Both the benchmark and the active strategy must be evaluated using the exact same end-of-test liquidation policy (either marked to market at the final bar's close or liquidated net of closing fees).

Key Quantitative Comparison MetricsCopy Anchor Link

When benchmarking an ETF trend filter against buy-and-hold, evaluate performance across risk, return, and exposure dimensions:

Complete Python code example: Testing a 200-day trend filter on an ETFCopy Anchor Link

The following Python script demonstrates how to construct, execute, and audit a 200-day Simple Moving Average trend filter on daily ETF data using OpenBacktesting v0.2.1.

The strategy logic follows the classic systematic regime rule:

• Buy Entry: When the bar close crosses above the 200-day SMA, enter a 95% equity long position on the next bar open (execution_timing="next_open").
• Cash Exit: When the bar close crosses below the 200-day SMA, close the position on the next bar open and sit in 100% cash.
• Friction: 5 basis points (0.05%) broker commission and 5 basis points execution slippage on all fills.

"""
ETF Trend-Following vs. Buy & Hold Benchmark in OpenBacktesting v0.2.1.

Strategy: 200-Day SMA Regime Filter.
Asset: Daily OHLCV Exchange-Traded Fund (ETF).
Frictions: 5 bps commission, 5 bps slippage, next-open execution.
"""

import numpy as np
import pandas as pd

from openbacktesting import Backtest, Strategy
from openbacktesting.broker.execution import Slippage


class SmaTrendFilterETFStrategy(Strategy):
    """
    Classic quantitative trend-following regime filter.
    
    Rules:
    - Precomputes a 200-period Simple Moving Average on closing prices.
    - Long Regime: Enter long when Close > 200-day SMA.
    - Risk-Off Regime: Liquidate to cash when Close < 200-day SMA.
    - Execution: All orders staged for next-bar open fill.
    """

    sma_period = 200

    def init(self):
        """Vectorized indicator precomputation across historical series."""
        # Calculate 200-period Simple Moving Average
        self.sma = self.ta.sma(self.close, length=self.sma_period)

    def next(self):
        """Bar-by-bar causal strategy execution."""
        # 1. Warm-up verification: ensure indicator has stabilized valid values
        if len(self.close) < self.sma_period + 2 or not self.sma.valid[-1]:
            return

        current_close = self.close[-1]
        sma_value = self.sma[-1]

        # 2. Risk-Off Filter: Exit to cash if price falls below the 200-day trend
        if self.position.is_long:
            if current_close < sma_value:
                self.position.close()

        # 3. Risk-On Filter: Enter long if price trades above the 200-day trend
        elif not self.position.is_long:
            if current_close > sma_value:
                # Allocate 95% of liquid equity, reserving 5% buffer for slippage & fees
                self.buy(percent=95)


def generate_sample_etf_data(seed: int = 42) -> pd.DataFrame:
    """
    Generate realistic multi-year daily ETF OHLCV data spanning multiple cycles.
    Includes secular bull runs, severe market drawdowns, and choppy consolidations.
    """
    np.random.seed(seed)
    # 10 years of business daily bars (approx 2,520 bars)
    dates = pd.date_range("2014-01-01", "2023-12-31", freq="B")
    n = len(dates)

    # Simulate realistic drift and volatility regimes
    drift = 0.00035  # ~8.8% annualized geometric drift
    volatility = 0.011  # ~17.5% annualized volatility
    daily_returns = np.random.normal(drift, volatility, size=n)

    # Inject realistic market drawdowns (e.g., severe correction shock)
    daily_returns[750:820] -= 0.0045   # Mid-cycle drawdown
    daily_returns[1550:1620] -= 0.0065 # Sharp market crash

    price_path = 100.0 * np.exp(np.cumsum(daily_returns))

    # Construct OHLC structure around simulated price
    noise_high = np.abs(np.random.normal(0, 0.006, size=n))
    noise_low = np.abs(np.random.normal(0, 0.006, size=n))
    high = price_path * (1.0 + noise_high)
    low = price_path * (1.0 - noise_low)
    
    # Intrabar open and close bounds
    open_price = low + (high - low) * np.random.uniform(0.2, 0.8, size=n)
    close_price = low + (high - low) * np.random.uniform(0.2, 0.8, size=n)
    volume = np.random.randint(15_000_000, 75_000_000, size=n)

    df = pd.DataFrame(
        {
            "Open": open_price,
            "High": high,
            "Low": low,
            "Close": close_price,
            "Volume": volume,
        },
        index=dates,
    )
    return df


def run_etf_comparison():
    """Configure, execute, and print comparative performance audit."""
    print("Initializing ETF historical dataset...")
    etf_data = generate_sample_etf_data()

    print(
        f"Data range: {etf_data.index[0].strftime('%Y-%m-%d')} to "
        f"{etf_data.index[-1].strftime('%Y-%m-%d')} ({len(etf_data)} daily bars)"
    )

    # Initialize Backtest engine
    bt = Backtest(
        data=etf_data,
        strategy=SmaTrendFilterETFStrategy,
        cash=100_000.0,
        commission=0.0005,                # 5 bps broker fee (0.05%)
        slippage=Slippage("bps", 5.0),    # 5 bps execution slippage model
        execution_timing="next_open",     # Causal order fill at next bar Open
        end_policy="close_at_last_close", # Settle open positions at final bar Close
    )

    # Execute simulation
    stats = bt.run()

    # Formatted Diagnostic Report
    print("\n" + "=" * 68)
    print("   ETF BACKTEST: 200-DAY TREND FILTER VS. BUY & HOLD BENCHMARK   ")
    print("=" * 68)
    
    comparison_table = [
        ("Simulation Duration", f"{stats['Duration'].days} days"),
        ("Active Strategy Return", f"{stats['Return [%]']:.2f}%"),
        ("Buy & Hold Benchmark Return", f"{stats['Buy & Hold Return [%]']:.2f}%"),
        ("Strategy CAGR", f"{stats['CAGR [%]']:.2f}%"),
        ("Strategy Max Drawdown", f"{stats['Max. Drawdown [%]']:.2f}%"),
        ("Annualized Sharpe Ratio", f"{stats['Sharpe Ratio']:.3f}"),
        ("Annualized Sortino Ratio", f"{stats['Sortino Ratio']:.3f}"),
        ("Calmar Ratio", f"{stats['Calmar Ratio']:.3f}"),
        ("Market Exposure Time", f"{stats['Exposure Time [%]']:.2f}%"),
        ("Total Trades Executed", f"{stats['# Trades']}"),
        ("Strategy Win Rate", f"{stats['Win Rate [%]']:.2f}%"),
        ("Profit Factor", f"{stats['Profit Factor']:.2f}"),
        ("Commissions & Fees Paid", f"${stats['Commissions [$]']:,.2f}"),
    ]

    for label, val in comparison_table:
        print(f"{label:<32}: {val:>32}")
    print("=" * 68)

    return stats


if __name__ == "__main__":
    run_etf_comparison()

Deconstructing the results: Return vs. Risk ReductionCopy Anchor Link

When quants review the performance output of a long-term ETF moving average backtest, the headline numbers frequently surprise beginners: the trend filter often generates a slightly lower or similar compound annual growth rate (CAGR) compared to buy-and-hold.

However, inspecting the multi-dimensional risk metrics reveals the true institutional objective of the strategy:

┌────────────────────────────────────────────────────────────────────────┐
│                   Performance Attribution Breakdown                    │
├──────────────────────────┬───────────────────────┬─────────────────────┤
│ Metric Dimension         │ Buy & Hold Benchmark  │ 200-Day SMA Filter  │
├──────────────────────────┼───────────────────────┼─────────────────────┤
│ Capital Compounding      │ High (Captures 100%)  │ Moderate to High    │
│ Downside Drawdown        │ Severe (-45% to -55%) │ Controlled (-15% to -25%)
│ Exposure Time            │ 100.0%                │ ~62.0% - 68.0%      │
│ Cash Drag / Idle Capital │ 0.0%                  │ ~32.0% - 38.0%      │
│ Calmar Ratio (Tail Risk) │ Baseline              │ Substantially Higher│
└──────────────────────────┴───────────────────────┴─────────────────────┘

1. Why CAGR is Frequently LowerCopy Anchor Link

The 200-day trend filter incurs two distinct mathematical costs:

• Whipsaw Cost in Sideways Ranges: When an ETF trades in a choppy, sideways consolidation around its 200-day mean, the strategy buys near local resistance and liquidates near local support. Each false breakout generates minor losses amplified by two rounds of commission and slippage.
• Lagged Re-entry in V-Shaped Recoveries: Following a steep market selloff, price rebounds sharply from the low. Because the 200-day SMA is a slow lagging indicator, the strategy remains in cash during the initial 15% to 25% recovery rally before confirming a bullish breakout.

2. The Tail-Risk Defense: Slashing Maximum DrawdownCopy Anchor Link

Where systematic beta filters excel is in capital preservation during structural secular bear markets.

During multi-year declines—such as the 2000–2002 Dot-Com crash (-50% for SPY, -82% for QQQ) or the 2007–2009 Subprime Crisis (-55% for SPY)—a 100% buy-and-hold investor experiences catastrophic drawdown. Recovering from a -50% drawdown requires a +100% gain merely to break even:

\text{Required Recovery Return} = \frac{1}{1 - \text{Drawdown}} - 1

$100,000 Portfolio Drawdown Recovery Asymmetry:
-20% Drawdown ──> Value = $80,000  ───> Requires +25.0% gain to reach peak
-35% Drawdown ──> Value = $65,000  ───> Requires +53.8% gain to reach peak
-50% Drawdown ──> Value = $50,000  ───> Requires +100.0% gain to reach peak
-70% Drawdown ──> Value = $30,000  ───> Requires +233.3% gain to reach peak

By exiting into cash when the ETF breaks below its long-term average, the trend filter caps maximum drawdowns to a fraction of the index's decline (typically -15% to -25%), completely sidestepping the severe tail-risk cascade.

3. Exposure Efficiency and the Power of CashCopy Anchor Link

A critical diagnostic in OpenBacktesting is Exposure Time [%]. In a typical 10-year run, the 200-day SMA filter spends approximately 62% to 68% of the time invested in the market.

This means that for roughly one-third of the decade, the portfolio is sitting entirely in risk-free cash:

• The strategy generated its returns while exposing capital to equity risk only two-thirds of the time.
• During risk-off regimes, unallocated cash reserves can earn the prevailing risk-free yield (e.g., 3-month Treasury bills) or serve as liquid margin collateral for non-correlated multi-asset systematic strategies.
• The Calmar Ratio (\text{CAGR} / |\text{Max Drawdown}|) and Sortino Ratio improve dramatically, demonstrating superior downside risk efficiency.

Common pitfalls in ETF backtestingCopy Anchor Link

Backtesting exchange-traded funds requires vigilance against several methodological pitfalls unique to pooled investment products.

                  ┌──────────────────────────────────────────────┐
                  │       Systematic ETF Backtesting Risks       │
                  └──────────────────────┬───────────────────────┘
                                         │
     ┌───────────────────┬───────────────┴───────────────┬───────────────────┐
     ▼                   ▼                               ▼                   ▼
Survivorship Bias   Short History                   Cash Drag Friction   Leveraged Decay
(Liquidated ETFs)   (Omitting Crises)               (Assumed 0% Yield)   (Volatility Drag)

1. Survivorship Bias in Thematic and Niche ETFsCopy Anchor Link

While broad index funds (SPY, QQQ) have existed for decades, hundreds of thematic, specialized sector, and commodity ETFs have closed or liquidated since 2000. If you backtest an asset allocation rule exclusively on today’s active thematic ETFs (e.g., clean energy, genomics, blockchain ETFs), you are testing only the subset of products that survived. Failed funds that suffered catastrophic collapses are excluded from current databases, creating an artificial upward bias in simulated returns.

2. Short Historical Sample DepthCopy Anchor Link

Attempting to validate an ETF trading system on a fund launched in 2021 provides fewer than 1,200 daily bars. This sample captures only a single monetary tightening cycle and cannot establish statistical significance. For newer ETFs, quantitative researchers often synthesize a composite proxy series by chaining the ETF's historical returns with the underlying index's total return history prior to the ETF's inception date.

3. Ignoring the Opportunity Cost of Cash (Cash Drag)Copy Anchor Link

When a trend filter exits an ETF, the capital sits in cash. In backtesting engines, cash typically earns a 0.0% nominal return. Over a multi-year bear market, assuming 0% yield during high interest-rate regimes (such as 1970–1982 or 2022–2024, where cash yielded 4% to 5%) slightly penalizes the active strategy relative to reality. Conversely, during zero-interest-rate policy (ZIRP) environments, assuming 0% yield is fully accurate.

4. Leveraged and Inverse ETF Decay (Beta Slippage)Copy Anchor Link

Applying simple moving averages or multi-month trend filters to 2x or 3x leveraged ETFs (TQQQ, UPRO, SOXL) is a severe quantitative mistake. Leveraged ETFs rebalance daily to maintain their stated leverage target. In volatile or oscillating markets, this constant daily rebalancing causes compounding volatility drag (also known as beta slippage):

R_{\text{leveraged}} \approx L \cdot R_{\text{index}} - \frac{1}{2} (L^2 - L) \sigma^2 \Delta t

Where L is the leverage factor and \sigma^2 is realized index variance. Because volatility drag continually erodes NAV during sideways chop, standard trend-following rules tested on leveraged ETFs without specialized intraday volatility dampers experience severe decay.

ETF Backtesting Verification MatrixCopy Anchor Link

Before deploying an ETF systematic strategy or accepting backtest metrics, complete this verification checklist:

Conclusion and resourcesCopy Anchor Link

Backtesting ETF strategies in Python allows quantitative researchers to systematically evaluate beta management hypotheses. By trading diversified baskets rather than single stocks, investors eliminate idiosyncratic bankruptcy risk and focus on the fundamental objective of quantitative investing: capturing broad market appreciation while systematically suppressing catastrophic drawdowns.

As demonstrated with OpenBacktesting v0.2.1, a classic 200-day trend filter on a major ETF does not necessarily outpace buy-and-hold in raw geometric return during prolonged bull markets. Instead, its primary value proposition lies in tail-risk truncation—slashing peak-to-trough drawdowns, boosting risk-adjusted metrics like the Sortino and Calmar ratios, and liberating capital into safe cash during turbulent market regimes.

To continue refining your quantitative backtesting workflows, explore these related technical resources:

• Stock Backtesting in Python: Data Adjustments, Costs and Benchmarks — Dive deep into single-stock nuances, corporate actions, and regulatory SEC/FINRA fees.
• Python Backtesting Tutorial: Data, Orders and Results — Step-by-step architectural breakdown of building inspectable trading strategies in OpenBacktesting.
• OpenBacktesting Official Documentation — Complete API reference for order execution models, position sizing, indicator pipelines, and local studio diagnostics.