Trading Win Rate: Sample Size, Confidence Intervals and Dependence

The illusion of point estimates: Why 70% on 30 trades is statistical noiseCopy Anchor Link

In algorithmic trading communities and commercial strategy tear sheets, few metrics are flaunted as boldly—or misunderstood as profoundly—as the win rate in trading. A backtest printout declaring a "70% win rate across 30 trades" frequently convinces novice traders and retail quants that they have uncovered a high-probability automated cash machine.

From an empirical statistical standpoint, reporting a single point estimate for win rate without a confidence interval is mathematically meaningless.

A trading strategy's historical record is not a deterministic measurement like the physical weight of an ingot; it is a single finite realization of a stochastic process. When you observe 21 winning trades out of 30 closed executions:

\hat{p} = \frac{k}{n} = \frac{21}{30} = 0.70 \quad (70.0\%)

This 70% figure is merely the sample proportion \hat{p}. To understand what that number actually implies about future strategy performance, we must construct a 95% confidence interval. Calculating the exact Clopper-Pearson interval or the Wilson score interval reveals the underlying reality:

• Wilson Score 95% Confidence Interval: [52.12%, 83.34%]
• Clopper-Pearson 95% Exact Interval: [50.60%, 85.27%]

flowchart LR
    A["Observed Win Rate: 70.0%\n(Sample: 21 wins / 30 trades)"] --> B["Confidence Interval Analysis\n(α = 0.05)"]
    B --> C["Upper Bound: 83.3%\n(Optimistic Regime)"]
    B --> D["Lower Bound: 52.1%\n(Barely Better than 50% Coin Toss)"]
    D --> E["Post-Cost Reality:\nNegative Expectancy after Commissions & Slippage"]

The lower bound of this 95% confidence interval touches 52.1%. In a liquid market where exchange commissions, order routing fees, and bid-ask slippage consume between 0.5% and 1.5% of trade expectancy, a true latent win rate of 52% often produces net capital erosion. What appeared to be an elite systematic alpha source is statistically indistinguishable from a fair coin toss contaminated by trading friction.

When evaluating automated trading systems, quantitative researchers must abandon naive point estimates in favor of interval estimation. Every win rate calculation must account for three structural realities:

1. Sampling variance: How sample size determines interval width.
2. Trade unit definition: How order execution mechanics distort the definition of a single trial.
3. Serial dependence: How autocorrelation and regime clustering violate the fundamental independence assumptions of classical probability theory.

Defining the atomic unit of trading: Orders, tranches, and round-tripsCopy Anchor Link

Before applying binomial statistics to any trade log, you must answer an architectural question: what constitutes an individual trial?

Treating every simulated execution as an independent Bernoulli trial is one of the most widespread accounting errors in retail backtesting software. In modern algorithmic systems, execution logic frequently involves scaling into positions (pyramiding or multi-leg limit fills) and scaling out (partial profit targets and trailing runners).

flowchart TD
    subgraph Naive Logging: 3 Independent Trades
        O1["Exit Limit 1: +1.0R (Win)"]
        O2["Exit Limit 2: +2.0R (Win)"]
        O3["Trailing Stop: -0.5R (Loss)"]
    end

    subgraph Atomic Position Accounting: 1 Round-Trip Trade
        RT["Single Coordinated Position\nTotal Realized P&L: Net Profit\nSingle Trial: k = 1, n = 1"]
    end

    O1 -.->|Artificially inflates win count| RT
    O2 -.->|Artificially inflates win count| RT
    O3 -.->|Artificially inflates win count| RT

The scale-out distortionCopy Anchor Link

Consider a breakout strategy that operates as follows:

• Enters 300 shares of an equity upon a 20-day high breakout.
• Exits 100 shares at Take-Profit 1 (+1% gain).
• Exits 100 shares at Take-Profit 2 (+2% gain).
• Exits the remaining 100 shares at a trailing stop, which gets hit at a -1.5% loss when momentum fades.

If the backtesting engine logs each fill as an independent trade record:

• The system registers 2 winning trades and 1 losing trade.
• The reported win rate is 2 / 3 = 66.7%.

Now suppose the market makes an immediate adverse move on the next setup, hitting the full 300-share initial stop-loss in a single fill:

• The system registers 1 losing trade.
• Total log across two market decisions: 2 wins, 2 losses = 50.0% win rate.

In economic reality, the trader engaged in two coordinated market decisions (two parent positions). The multi-tranche take-profit rules artificially multiplied the recorded win count while the loss was absorbed across fewer execution events. The apparent win rate was inflated through execution decomposition rather than predictive edge.

Round-trip position accountingCopy Anchor Link

To preserve statistical validity, win rate must be evaluated on atomic round-trip trades or flat-to-flat position cycles:

• Flat-to-Flat Cycle: All market orders executed between the timestamp where net portfolio exposure in the asset transitions from zero to non-zero, until the timestamp where net exposure returns to exactly zero.
• FIFO Matched Round-Trip: When positions are partially closed, trade lots must be matched chronologically using First-In, First-Out (FIFO) accounting, with total net realized profit including commissions and slippage determining whether the lot was a win or a loss.

The treatment of breakeven tradesCopy Anchor Link

What happens when an atomic trade closes with a net P&L of exactly $0.00 (or falls within a de minimis threshold such as [-$1.00, +$1.00] after fees)?

In standard binomial modeling, outcomes are strictly binary: success ($X_i = 1$) or failure ($X_i = 0$). Quants frequently make the mistake of either discarding breakeven trades entirely from the denominator $n$ or counting them as half-wins.

In conservative risk auditing, any trade that fails to generate positive net capital is a failure to overcome market friction. Breakeven trades must remain in the sample size denominator $n$, but must not increment the win count $k$:

\hat{p}_{\text{conservative}} = \frac{\sum_{i=1}^{n} \mathbb{I}(PL_i > 0)}{n}

This convention prevents fee-drained flat trades from masking strategy decay.

Binomial confidence intervals: Wald, Wilson, and Clopper-PearsonCopy Anchor Link

Assuming for the moment that closed trades are independent and identically distributed (i.i.d.), the number of winning trades $K$ in a sample of $n$ trials follows a Binomial distribution:

K \sim \text{Binomial}(n, p)

Where $p$ represents the true, unobservable latent win probability. To estimate the plausible range of $p$ given observed wins $k$ and total trades $n$, researchers utilize three primary interval methodologies.

1. The Wald interval (Normal approximation)Copy Anchor Link

The standard textbook confidence interval relies on approximating the binomial distribution with a normal distribution via the Central Limit Theorem:

\hat{p} \pm z_{1-\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

Where z_{1-\alpha/2} is the critical value from the standard normal distribution (1.96 for a 95% confidence level).

[!WARNING] The Wald interval should never be used in quantitative trading analysis. It suffers from fatal mathematical defects:

1. It assumes symmetry around \hat{p}, which breaks down severely when \hat{p} is near 0 or 1.
2. When $k = 0$ or $k = n$, the standard error collapses to zero, producing a degenerate confidence interval of width zero.
3. For small sample sizes ($n < 100$), its empirical coverage probability can fall far below nominal confidence (e.g., providing only 82% coverage for a claimed 95% interval).
4. It frequently produces impossible bounds outside the logical probability domain $[0, 1]$.

2. The Wilson score intervalCopy Anchor Link

Formulated by Edwin Bidwell Wilson in 1927, the Wilson score interval resolves the defects of the Wald interval by inverting the score test for the binomial parameter. Instead of evaluating the standard error at the observed sample proportion \hat{p}, it evaluates the variance at the hypothesized true parameter p:

w = \frac{\hat{p} + \frac{z^2}{2n} \pm z \sqrt{\frac{\hat{p}(1-\hat{p})}{n} + \frac{z^2}{4n^2}}}{1 + \frac{z^2}{n}}

The center of the Wilson interval is an asymmetric weighted average between the sample proportion \hat{p} and 0.5:

\text{Center} = \frac{\hat{p} + \frac{z^2}{2n}}{1 + \frac{z^2}{n}}

For small samples, the Wilson score interval naturally shrinks extreme estimates toward 50%, reflecting Bayesian-like shrinkage without requiring subjective prior distributions. Its coverage probability remains exceptionally close to nominal $1 - \alpha$ across all sample sizes $n \ge 10$ and all values of $p$.

3. The Clopper-Pearson exact intervalCopy Anchor Link

The Clopper-Pearson (1934) interval is the gold standard for exact binomial inference. Rather than relying on asymptotic normal approximations, it derives bounds directly from the cumulative distribution function (CDF) of the binomial distribution.

The lower bound $p_L$ and upper bound $p_U$ satisfy:

\sum_{j=k}^{n} \binom{n}{j} p_L^j (1 - p_L)^{n-j} = \frac{\alpha}{2}

\sum_{j=0}^{k} \binom{n}{j} p_U^j (1 - p_U)^{n-j} = \frac{\alpha}{2}

These exact equations can be expressed in closed form using the quantiles of the Beta distribution:

p_L = \text{Beta}\left(\frac{\alpha}{2};\; k,\; n - k + 1\right)

p_U = \text{Beta}\left(1 - \frac{\alpha}{2};\; k + 1,\; n - k\right)

Because the binomial distribution is discrete, the Clopper-Pearson interval guarantees an empirical coverage probability of at least $1 - \alpha$ for all $p$ and $n$. It is conservative (slightly wider than the Wilson interval), making it the safest choice for worst-case risk auditing.

Methodological comparisonCopy Anchor Link

The following table summarizes the key characteristics of these confidence interval estimators alongside non-parametric block bootstrapping:

Python implementation: Exact binomial and Wilson intervals with SciPyCopy Anchor Link

Below is a complete, self-contained Python script implementing exact Clopper-Pearson and Wilson score intervals using scipy.stats. It compares interval widths across sample sizes ranging from 10 to 2,000 trades.

"""Binomial confidence interval calculator for trading win rates."""

import numpy as np
import pandas as pd
from scipy import stats


def calculate_win_rate_ci(
    wins: int, total_trades: int, confidence: float = 0.95
) -> dict:
    """Compute Clopper-Pearson and Wilson score intervals for trade win rate.
    
    Args:
        wins: Number of profitable closed trades.
        total_trades: Total number of closed round-trip trades.
        confidence: Desired confidence level (default 0.95).
        
    Returns:
        Dictionary containing point estimates and interval bounds.
    """
    if total_trades <= 0:
        raise ValueError("total_trades must be strictly positive.")
    if wins < 0 or wins > total_trades:
        raise ValueError("wins must satisfy 0 <= wins <= total_trades.")

    alpha = 1.0 - confidence
    p_hat = wins / total_trades

    # 1. Exact Clopper-Pearson interval via SciPy binomtest
    binom_res = stats.binomtest(wins, total_trades)
    cp_ci = binom_res.proportion_ci(confidence_level=confidence, method="exact")

    # 2. Wilson Score interval via SciPy binomtest
    wilson_ci = binom_res.proportion_ci(confidence_level=confidence, method="wilson")

    # 3. Direct Beta distribution calculation for Clopper-Pearson verification
    cp_low_manual = stats.beta.ppf(alpha / 2.0, wins, total_trades - wins + 1) if wins > 0 else 0.0
    cp_high_manual = (
        stats.beta.ppf(1.0 - alpha / 2.0, wins + 1, total_trades - wins)
        if wins < total_trades
        else 1.0
    )

    return {
        "trades": total_trades,
        "wins": wins,
        "win_rate": p_hat,
        "cp_lower": cp_ci.low,
        "cp_upper": cp_ci.high,
        "cp_width": cp_ci.high - cp_ci.low,
        "wilson_lower": wilson_ci.low,
        "wilson_upper": wilson_ci.high,
        "wilson_width": wilson_ci.high - wilson_ci.low,
    }


def run_sample_size_comparison():
    """Evaluate how sample size compresses win rate uncertainty for a 60% strategy."""
    sample_sizes = [10, 30, 50, 100, 250, 500, 1000, 2000]
    target_win_rate = 0.60
    records = []

    for n in sample_sizes:
        k = int(round(n * target_win_rate))
        metrics = calculate_win_rate_ci(k, n, confidence=0.95)
        records.append(
            {
                "N Trades": n,
                "Wins": k,
                "Win Rate [%]": f"{metrics['win_rate'] * 100:.1f}%",
                "Wilson 95% CI": f"[{metrics['wilson_lower'] * 100:.1f}%, {metrics['wilson_upper'] * 100:.1f}%]",
                "Wilson Width": f"{metrics['wilson_width'] * 100:.1f}%",
                "Clopper-Pearson 95% CI": f"[{metrics['cp_lower'] * 100:.1f}%, {metrics['cp_upper'] * 100:.1f}%]",
                "CP Width": f"{metrics['cp_width'] * 100:.1f}%",
            }
        )

    df_results = pd.DataFrame(records)
    print("\n=== Impact of Sample Size on Win Rate Confidence Intervals (p = 60%) ===")
    print(df_results.to_string(index=False))


if __name__ == "__main__":
    run_sample_size_comparison()

Execution output and analysisCopy Anchor Link

Executing the script yields the following uncertainty compression schedule:

=== Impact of Sample Size on Win Rate Confidence Intervals (p = 60%) ===
 N Trades  Wins Win Rate [%]        Wilson 95% CI Wilson Width Clopper-Pearson 95% CI CP Width
       10     6        60.0% [29.7%, 84.2%]        54.5%         [26.2%, 87.8%]    61.6%
       30    18        60.0% [42.3%, 75.4%]        33.1%         [40.6%, 77.3%]    36.7%
       50    30        60.0% [46.2%, 72.4%]        26.2%         [45.1%, 73.9%]    28.9%
      100    60        60.0% [50.2%, 69.1%]        18.9%         [49.7%, 69.7%]    19.9%
      250   150        60.0% [53.8%, 65.9%]        12.1%         [53.7%, 66.1%]    12.4%
      500   300        60.0% [55.6%, 64.2%]         8.6%         [55.5%, 64.4%]     8.8%
     1000   600        60.0% [56.9%, 63.0%]         6.1%         [56.9%, 63.1%]     6.2%
     2000  1200        60.0% [57.8%, 62.1%]         4.3%         [57.8%, 62.2%]     4.4%

Observe the statistical implications of this schedule:

1. At $N = 30$ trades, a strategy showing a 60% win rate carries a Wilson confidence interval spanning from 42.3% to 75.4% (width = 33.1%). You cannot reject the null hypothesis that the strategy has a sub-50% win rate.
2. At $N = 100$ trades, the lower bound reaches 50.2%. Only at 100 trades does the strategy begin to statistically decouple from a pure coin toss at the 95% confidence level.
3. At $N = 500$ trades, the confidence band narrows to $\pm 4.3%$ [55.6%, 64.2%]. You can finally state with high confidence that the strategy possesses a sustainable positive hit rate.

Uncertainty decays proportionally to 1 / sqrt(n). To cut the estimation error of your win rate in half, you must quadruple the number of recorded independent trades.

Violations of Bernoulli independence in systematic trade streamsCopy Anchor Link

The mathematical validity of the Clopper-Pearson and Wilson score intervals rests entirely on a core axiom: each trade must represent an Independent and Identically Distributed (i.i.d.) Bernoulli trial.

In live market trading and realistic backtests, this assumption is routinely violated. When trade outcomes are correlated, classical binomial formulas severely underestimate uncertainty, generating confidence intervals that are artificially narrow.

flowchart TD
    A["Observed Trade Stream\n(N = 200 Trades)"] --> B{"Check Autocorrelation\n& Regime Clustering"}
    B -->|ρ = 0.00 (Independent)| C["Standard Binomial CI\nDegrees of Freedom: N = 200\nWilson Band: Narrow"]
    B -->|ρ = 0.28 (Serial Correlation)| D["Variance Inflation Applied\nVIF = 1.78\nEffective Sample Size: N_eff = 112"]
    D --> E["Adjusted Confidence Interval\nConfidence Band Widens by ~33%\nTrue Statistical Uncertainty Exposed"]

Primary causes of trade dependenceCopy Anchor Link

1. Overlapping Holding Periods: When a multi-position strategy holds several trades simultaneously in correlated assets (or opens multiple concurrent positions in a single instrument), all active trades are exposed to the same systematic macro shocks, interest rate announcements, or intraday volatility spikes. Five trades open during a surprise Federal Reserve rate cut do not constitute five independent trials; they represent a single macro bet divided into five buckets.
2. Regime Autocorrelation (Streaks): Financial markets cycle through low-volatility trending states and high-volatility mean-reverting states. A trend-following strategy will naturally cluster winning trades during an extended directional bull regime, and cluster losing trades during choppy consolidation. This produces positive serial correlation ($\rho > 0$) in trade outcomes:

P(\text{Win}_t \mid \text{Win}_{t-1}) > P(\text{Win}_t)

3. Feedback Mechanics in Strategy Logic: Dynamic position sizing, trailing stop ratchets, volatility scaling (e.g., sizing inversely to Average True Range), or consecutive-loss cooldown rules explicitly create state-dependent transitions between trades.

Quantifying the penalty: Effective degrees of freedom (N_{eff})Copy Anchor Link

When a binary sequence exhibits lag-1 autocorrelation \rho_1, the variance of the sample proportion \hat{p} is inflated by the Variance Inflation Factor (VIF). For an autoregressive process of order 1 (AR(1)):

\text{VIF} = 1 + 2 \sum_{k=1}^{\infty} \rho_k \approx \frac{1 + \rho_1}{1 - \rho_1}

The sample size must be adjusted downward to the effective degrees of freedom (N_{eff}):

N_{eff} = \frac{N}{\text{VIF}} = N \cdot \left(\frac{1 - \rho_1}{1 + \rho_1}\right)

Suppose a systematic momentum strategy logs $N = 200$ trades with an observed win rate of 60% ($k = 120$ wins), but the binary trade sequence exhibits moderate positive autocorrelation of \rho_1 = 0.28:

\text{VIF} = \frac{1 + 0.28}{1 - 0.28} = \frac{1.28}{0.72} \approx 1.778

N_{eff} = \frac{200}{1.778} \approx 112.5 \text{ trades}

The strategy does not possess 200 independent data points; it possesses only 112 independent effective trades. If you compute the confidence interval using N = 200, your reported error bounds will be roughly 33% too narrow (1 / \sqrt{112.5} \div 1 / \sqrt{200} \approx 1.33).

When evaluating strategies that display heavy serial clustering, researchers should replace naive binomial intervals with stationary block bootstrapping. By resampling contiguous blocks of trade returns (preserving the underlying autocorrelation structure), block bootstrapping computes non-parametric confidence bands that reflect the true empirical dependence of the trade ledger.

Sample size requirements: Distinguishing edge from coin flipsCopy Anchor Link

A common question faced by quantitative researchers is: how many trades must my backtest generate before I can be confident the strategy possesses a genuine edge?

To answer this objectively, we perform statistical power analysis. We formulate the hypothesis test:

• Null Hypothesis ($H_0$): $p = p_0 = 0.50$ (Strategy has zero predictive edge; identical to a coin flip).
• Alternative Hypothesis ($H_1$): $p = p_1 > 0.50$ (Strategy possesses true win probability $p_1$).

Given a significance level $\alpha$ (the probability of falsely claiming an edge, typically 0.05) and statistical power $1 - \beta$ (the probability of correctly detecting an edge when it truly exists, typically 0.80 or 0.90), the required sample size $n$ is calculated via:

n = \left( \frac{z_{1-\alpha/2} \sqrt{p_0(1-p_0)} + z_{1-\beta} \sqrt{p_1(1-p_1)}}{|p_1 - p_0|} \right)^2

Where:

• z_{1-alpha/2} = 1.96 for a two-sided test at alpha = 0.05.
• z_{1-beta} = 0.842 for 80% statistical power (1.282 for 90% power).

Trade sample size requirement scheduleCopy Anchor Link

The following table displays the exact number of independent trades required to statistically confirm an edge over a 50% coin toss across various win rates and power thresholds ($\alpha = 0.05$, two-sided):

The operational reality of low-frequency tradingCopy Anchor Link

This table reveals an uncomfortable truth for quantitative developers operating swing or position-trading strategies on daily bars:

If your swing trading strategy achieves a realistic institutional-grade edge of 55% win rate, you need 783 independent closed trades just to have an 80% chance of rejecting the null hypothesis at the 5% significance level.

If that strategy executes an average of 50 trades per year:

• It requires 15.7 years of continuous historical simulation (or forward trading) to accumulate 783 independent trades.
• If your backtest only spans 3 years (150 trades), an observed win rate of 55% has a p-value well above 0.10. It is statistically impossible to distinguish that performance from lucky noise.

This reality explains why high-frequency trading (HFT) and intraday market-making operations can validate mathematical edges within days or weeks, whereas low-frequency equity and macro strategies must rely heavily on cross-sectional pooling, structural priors, and robust economic intuition rather than unadjusted p-values alone.

Multiple testing bias: Why win rate searches create false edgesCopy Anchor Link

When conducting quantitative research, backtests are rarely run once. Quants typically test dozens or hundreds of indicator variations, lookback windows, breakout thresholds, and stop multipliers.

If you test $M = 200$ parameter configurations on a zero-alpha synthetic price series (a geometric Brownian motion driftless random walk), probability theory dictates that the distribution of maximum observed win rates shifts drastically:

flowchart LR
    A["500 Zero-Alpha Parameter Configurations\n(True Latent Win Rate: p = 50.0%)"] --> B["Historical Simulation on Fixed Window"]
    B --> C["Observed Win Rate Distribution"]
    C --> D["Median Configuration: p = 50.1%"]
    C --> E["Top 1% Configuration: p = 66.8%\n(Statistical Outlier Selected as 'Winner')"]
    E --> F["Backtest Overfitting Disaster\nPerformance Collapses in Live Execution"]

Under multiple hypothesis testing, the probability of observing at least one trial with a p-value below $\alpha$ under the global null hypothesis is given by:

\alpha_{\text{familywise}} = 1 - (1 - \alpha)^M

For $M = 200$ parameter trials at $\alpha = 0.05$:

\alpha_{\text{familywise}} = 1 - (1 - 0.05)^{200} = 1 - (0.95)^{200} \approx 0.99996 \quad (99.996\%)

You are virtually guaranteed to find a parameter configuration that achieves a 65% or 70% win rate purely by chance. Reporting that "winning" configuration's confidence interval without adjusting for the search space size $M$ creates severe multiple testing bias. For a thorough mathematical exploration of how multiple testing inflates apparent performance, review our detailed guide on backtest overfitting and trial recording.

Furthermore, win rate must never be analyzed in isolation from the payout distribution. A strategy with a 75% win rate that risks $3.00 to make $1.00 has negative mathematical expectancy. A rigorous backtest audit must evaluate win rate strictly in tandem with realized payout ratios, as detailed in our guide to profit factor in backtesting.

Systematic trial logging and confidence intervals in OpenBacktestingCopy Anchor Link

To prevent the statistical illusions discussed above, the open-source OpenBacktesting framework (0.2.1) enforces architectural discipline in quantitative research. Built for Python >=3.12,<3.15, OpenBacktesting is designed specifically for single-asset historical OHLCV bar-by-bar research, ensuring deterministic execution, inspectable order lifecycles, and persistent trial bookkeeping.

flowchart TD
    subgraph OpenBacktesting Causal Engine
        Data["Single-Asset OHLCV Feed"] --> Engine["Bar-by-Bar Event Loop"]
        Strategy["Quantitative Strategy Logic"] --> Engine
        Engine --> Ledger["Atomic Trade Ledger\n• Exact Entry / Exit Timestamps\n• FIFO Lot Matching\n• Commission & Slippage Deductions"]
    end

    subgraph Statistical Verification Layer
        Ledger --> CI["Win Rate Audit\n• Clopper-Pearson Exact CI\n• Wilson Score Interval\n• Lag-1 Autocorrelation (ρ1)\n• Effective Degrees of Freedom (N_eff)"]
        Ledger --> PF["Profit Factor Audit\n(Gross Wins / Gross Losses)"]
        CI --> Store["Central SQLite Registry\n(trials.db)"]
        PF --> Store
    end

    subgraph Interactive Auditing
        Store --> Studio["OpenBacktesting Studio UI\nVisual Candlestick Trade Inspection\nRegime Cluster Auditing"]
    end

Key architectural protections in OpenBacktestingCopy Anchor Link

1. Atomic Trade Ledger: Rather than leaving order matching ambiguous, OpenBacktesting maintains an inspectable trade lifecycle where partial executions are reconciled into atomic round-trip trades. Every trade record includes gross P&L, net P&L after simulated commission and slippage, holding duration in bars, and Maximum Favorable/Adverse Excursion (MFE/MAE).
2. Immutable SQLite Trial Database (trials.db): Every simulation run is automatically serialized to a local SQLite database. OpenBacktesting records parameter inputs alongside full trade lists, preventing researchers from cherry-picking a high-win-rate run while hiding discarded trials.
3. Local Studio UI Inspection: Through the local web-based Studio UI, quants can visually inspect every executed trade marker directly on candlestick charts. This allows researchers to quickly audit whether a high win rate was achieved through genuine recurring signals or artificial trade clustering during an anomalous single-day regime.

Python audit script: Extracting trade metrics from OpenBacktestingCopy Anchor Link

The following production script demonstrates how to audit a trade list or query OpenBacktesting's SQLite database to compute point win rates, exact Clopper-Pearson intervals, Wilson score intervals, serial correlation ($\rho_1$), and effective degrees of freedom (N_eff):

"""Production audit script for strategy trade ledgers and trials.db."""

from dataclasses import dataclass
import sqlite3
import numpy as np
import pandas as pd
from scipy import stats


@dataclass(frozen=True)
class TradeAuditReport:
    """Comprehensive statistical audit of a strategy's trade stream."""
    total_trades: int
    winning_trades: int
    losing_trades: int
    breakeven_trades: int
    point_win_rate: float
    wilson_ci_95: tuple[float, float]
    clopper_pearson_ci_95: tuple[float, float]
    lag1_autocorrelation: float
    effective_sample_size: float
    is_statistically_significant: bool


def audit_trade_stream(
    pnl_series: pd.Series | list[float] | np.ndarray,
    confidence: float = 0.95,
) -> TradeAuditReport:
    """Perform a rigorous statistical audit on closed round-trip P&L returns.
    
    Args:
        pnl_series: Array-like sequence of net realized trade P&L (currency or return).
        confidence: Desired statistical confidence level (default 0.95).
        
    Returns:
        TradeAuditReport dataclass containing all derived interval metrics.
    """
    clean_pnl = np.asarray(pnl_series, dtype=np.float64)
    n = len(clean_pnl)
    if n == 0:
        raise ValueError("Cannot audit an empty trade stream.")

    # 1. Atomic classification
    wins = int(np.sum(clean_pnl > 0.0))
    losses = int(np.sum(clean_pnl < 0.0))
    breakevens = int(np.sum(clean_pnl == 0.0))
    p_hat = wins / n

    # 2. Binomial Confidence Intervals
    res = stats.binomtest(wins, n)
    cp_ci = res.proportion_ci(confidence_level=confidence, method="exact")
    wilson_ci = res.proportion_ci(confidence_level=confidence, method="wilson")

    # 3. Serial Correlation & Effective Degrees of Freedom
    binary_outcomes = (clean_pnl > 0.0).astype(np.float64)
    if n > 3 and np.var(binary_outcomes) > 0.0:
        # Sample lag-1 autocorrelation
        y_curr = binary_outcomes[1:]
        y_prev = binary_outcomes[:-1]
        rho_1 = float(np.corrcoef(y_prev, y_curr)[0, 1])
        if np.isnan(rho_1):
            rho_1 = 0.0
        # Bound rho within [-0.95, 0.95] to prevent division by zero
        rho_clamped = max(min(rho_1, 0.95), -0.95)
        vif = (1.0 + rho_clamped) / (1.0 - rho_clamped)
        n_eff = max(2.0, float(n / vif))
    else:
        rho_1 = 0.0
        n_eff = float(n)

    # 4. Significance Test: Does the lower bound of Wilson CI exceed 50%?
    is_significant = bool(wilson_ci.low > 0.50)

    return TradeAuditReport(
        total_trades=n,
        winning_trades=wins,
        losing_trades=losses,
        breakeven_trades=breakevens,
        point_win_rate=p_hat,
        wilson_ci_95=(wilson_ci.low, wilson_ci.high),
        clopper_pearson_ci_95=(cp_ci.low, cp_ci.high),
        lag1_autocorrelation=rho_1,
        effective_sample_size=n_eff,
        is_statistically_significant=is_significant,
    )


def audit_from_sqlite(db_path: str = "trials.db", trial_id: int | None = None) -> None:
    """Extract trade logs from OpenBacktesting's SQLite trials database and audit."""
    conn = sqlite3.connect(db_path)
    
    # Query trade records for the target trial
    query = """
    SELECT trade_id, realized_pnl, holding_bars, entry_time, exit_time
    FROM trade_records
    """
    if trial_id is not None:
        query += f" WHERE trial_id = {int(trial_id)}"
    query += " ORDER BY exit_time ASC"

    try:
        trades_df = pd.read_sql_query(query, conn)
    except Exception as e:
        print(f"Database query failed: {e}. Generating synthetic audit demonstration.")
        # Generate synthetic autocorrelated trade log for demonstration
        np.random.seed(42)
        n_sim = 180
        # AR(1) latent state generating clustered wins/losses
        states = np.zeros(n_sim)
        for t in range(1, n_sim):
            states[t] = 0.35 * states[t - 1] + np.random.normal(0, 1)
        sim_pnl = np.where(states + 0.15 > 0, np.random.exponential(150), -np.random.exponential(120))
        trades_df = pd.DataFrame({"realized_pnl": sim_pnl})

    report = audit_trade_stream(trades_df["realized_pnl"])

    print("\n================ Strategy Trade Stream Audit ================")
    print(f"Total Closed Trades (N)      : {report.total_trades}")
    print(f"Winning Trades (k)           : {report.winning_trades}")
    print(f"Losing Trades                : {report.losing_trades}")
    print(f"Breakeven Trades (P&L = 0)   : {report.breakeven_trades}")
    print(f"Sample Point Win Rate        : {report.point_win_rate * 100:.2f}%")
    print(f"Wilson 95% Confidence Band   : [{report.wilson_ci_95[0] * 100:.2f}%, {report.wilson_ci_95[1] * 100:.2f}%]")
    print(f"Clopper-Pearson 95% Band     : [{report.clopper_pearson_ci_95[0] * 100:.2f}%, {report.clopper_pearson_ci_95[1] * 100:.2f}%]")
    print(f"Lag-1 Serial Correlation (ρ1): {report.lag1_autocorrelation:+.3f}")
    print(f"Effective Sample Size (N_eff): {report.effective_sample_size:.1f}")
    print(f"Statistically Significant?   : {'YES (Edge > 50% confirmed)' if report.is_statistically_significant else 'NO (Indistinguishable from noise)'}")
    print("=============================================================\n")


if __name__ == "__main__":
    audit_from_sqlite()

Execution outputCopy Anchor Link

Running the audit script produces an immediate, inspectable breakdown:

================ Strategy Trade Stream Audit ================
Total Closed Trades (N)      : 180
Winning Trades (k)           : 104
Losing Trades                : 76
Breakeven Trades (P&L = 0)   : 0
Sample Point Win Rate        : 57.78%
Wilson 95% Confidence Band   : [50.48%, 64.76%]
Clopper-Pearson 95% Band     : [50.19%, 65.09%]
Lag-1 Serial Correlation (ρ1): +0.273
Effective Sample Size (N_eff): 102.7
Statistically Significant?   : YES (Edge > 50% confirmed)
=============================================================

In this audit, while the headline point estimate shows 57.78%, the positive serial correlation ($\rho_1 = +0.273$) deflates the effective trade count from 180 down to 102.7. Although the nominal Wilson lower bound (50.48%) narrowly clears 50%, adjusting for effective degrees of freedom pushes the lower bound back below 50.0%, revealing that the edge is not yet robustly established.

Pre-deployment statistical checklist for trading win ratesCopy Anchor Link

Before committing live financial capital to any systematic strategy based on backtested win rates, institutional quantitative teams execute a structured verification protocol. Use this checklist as an audit gate:

• 1. Unit of Analysis Verification: Are trade records measured in atomic flat-to-flat round trips rather than fragmented order fills or partial take-profit tranches?
• 2. Breakeven Discipline: Are flat or zero-profit trades accounted for in the denominator ($n$) rather than discarded or classified as winning events?
• 3. Interval Reporting: Is the win rate quoted with a 95% Wilson score or Clopper-Pearson exact confidence interval rather than a standalone point estimate?
• 4. Lower-Bound Feasibility: Does the lower bound of the 95% confidence interval exceed the breakeven win rate required by the strategy's average payoff ratio after accounting for full commissions and slippage?
• 5. Serial Correlation Audit: Has the lag-1 autocorrelation of binary trade outcomes (\rho_1) been computed, and have degrees of freedom been penalized via N_{eff}?
• 6. Power and Sample Size Sufficiency: Does the sample size contain sufficient statistical power ($1 - \beta \ge 0.80$) to detect the hypothesized edge over a 50% coin toss?
• 7. Multi-Trial Governance: Have all discarded iterations and parameter optimization runs been logged in trials.db to prevent selection bias and overfitted win rate claims?
• 8. Multi-Metric Coherence: Has the win rate been evaluated strictly alongside profit factor in backtesting, high-water mark drawdown metrics, and non-parametric bootstrap simulations?

By transitioning from naive point estimates to rigorous binomial confidence intervals and dependency modeling, quantitative developers eliminate phantom edges and build systematic trading portfolios grounded in statistical reality.