# 🔍 Reading Results and Avoiding Overfitting

By Anh Le

In the last lesson you built a backtest that charges realistic fills, slippage, and fees. That gives you a number. This lesson is about whether that number means anything. Most beginner strategies look brilliant on the first run and then fall apart with real money, and the reason is almost always the same: the rules were fitted to the past rather than tested against it.

By the end of this lesson you will be able to:

- Read an equity curve for shape, drawdowns, and concentration of profit
- Read a trade log to find where the money actually comes from
- Pick a small set of metrics and know what each one hides
- Set up a walk-forward split so some data stays untouched
- Run parameter sanity checks that expose fragile, over-fitted rules

## What the Equity Curve Is Telling You

The equity curve is cumulative account value over time. Beginners look at the end point; professionals look at the shape.

Four things to check, in order:

1. **Slope and consistency.** Is the rise steady, or does it come in a few vertical jumps? A smooth 45-degree line is healthier than a staircase with three big steps.
2. **Drawdowns.** Find every peak-to-trough decline. The largest one, _____, is the number that decides whether you could actually sit through the strategy. If it is 40% and you would panic at 15%, the strategy is not tradeable for you regardless of its return.
3. **Flat spots.** Long horizontal stretches mean the edge is regime-dependent. Note when they happen.
4. **Time in market.** A curve that rises only while you hold positions tells you the return is not from sitting in cash.

> **Tip:** 
> Plot the equity curve on a log scale once your account grows. On a linear chart, later dollar gains look enormous even when the percentage gain is identical.

**Try it on your capstone.** Your curve ends at 2.3x over ten years, but 60% of the total profit comes from three trades out of 220. What do you check next?

**Click to reveal the answer**

Check concentration and regime. Do those three trades share a date range, one symbol, or one volatility event? Re-run the backtest with them removed. If the edge disappears, you do not have a strategy, you have three lucky days.

## Reading the Trade Log Line by Line

The trade log is one row per round trip: entry date, exit date, direction, size, entry price, exit price, fees paid, and net profit. Reading it as a list of numbers misses the point. Read it as a distribution.

- **Win rate alone is meaningless.** A 90% win rate with one catastrophic loser can lose money. What matters is _____, the average profit per trade: win rate times average win, minus loss rate times average loss.
- **Payoff ratio.** Average win divided by average loss. Trend systems often win 35% of the time with a 3:1 payoff, and that is a perfectly good business.
- **Largest winners as a share of total profit.** Above roughly 30% from a handful of trades, your result depends on luck.
- **Turnover and costs.** If fees and slippage eat more than a third of gross profit, the strategy is fragile: a small cost increase kills it.
- **Holding period.** Very short holds are far more sensitive to slippage than the model assumed.
- **Trade count.** With fewer than about 30 trades, treat every statistic as a rumor.

**Try it on your capstone.** Your log shows 180 trades, 55% win rate, average win \$120, average loss \$140. Is this profitable per trade?

**Click to reveal the answer**

Expectancy = 0.55 × 120 − 0.45 × 140 = 66 − 63 = **\$3 per trade**. Positive, but so thin that a few dollars of extra slippage erases it. Before celebrating, check whether your cost assumptions were optimistic.

## Metrics Worth Reporting

Keep the dashboard small. These are enough for a capstone:

| Metric | What it tells you | What it hides |
| --- | --- | --- |
| CAGR | Compounded annual growth | Path taken to get there |
| Annualized volatility | Size of the ride | Direction of returns |
| Sharpe ratio | Return per unit of total risk | Fat tails, asymmetry |
| Max drawdown | Worst peak-to-trough loss | How long it lasted |
| Hit rate and payoff ratio | Shape of the trade distribution | Sample size problems |
| Turnover | Trading intensity and cost drag | Per-trade execution quality |

Sharpe ratio is the one people over-trust. It assumes roughly normal returns and treats upside and downside volatility the same. A strategy that makes small steady gains and occasionally loses everything can post an excellent Sharpe right up to the day it fails. Always read Sharpe next to max drawdown.

## Overfitting: When the Backtest Is Too Good

_____ means your rules, and especially your parameter values, describe the noise in one historical sample rather than a repeatable edge. Your backtest becomes a very accurate description of the past and a poor guide to the future.

Warning signs:

- Many parameters relative to the number of trades (a rough ceiling is one parameter per 30 to 50 trades)
- Rules that reference specific dates, news events, or one symbol's quirks
- In-sample Sharpe of 2.5 and out-of-sample Sharpe near zero
- Performance that collapses when you shift the start date by a month
- No economic reason why the edge should exist

The defense is not a better optimizer. It is **withholding data**.

## Walk-Forward Testing on Data You Did Not Touch

In walk-forward testing you _____. Only the stitched out-of-sample results count as evidence.

```mermaid
flowchart TD
  A["🗓️ Full price history"] --> B["In-sample window<br/>choose parameters"]
  B --> C["Out-of-sample window<br/>test once, no peeking"]
  C --> D["Roll both windows forward"]
  D --> B
  C --> E["📊 Stitched out-of-sample curve"]
```

Two practical choices: an *anchored* split keeps the training window growing from the start, while a *rolling* split keeps it fixed length and lets old data drop off. Rolling adapts faster; anchored uses more data. Pick one, state it in your write-up, and do not tune it after seeing the result.

**Try it on your capstone.** Six parameters optimized over 2015 to 2019 give an in-sample Sharpe of 2.4. The following year gives 0.3. What is your diagnosis and next move?

**Click to reveal the answer**

Classic overfitting: too many degrees of freedom for the sample. Cut to two or three parameters, simplify the entry rule, re-run the walk-forward with several folds, and judge the strategy on the stitched out-of-sample curve, not the best fold.

## Parameter Sanity Checks: Plateaus Beat Spikes

A single optimized value is a coincidence until you look around it.

- **Plateau test.** Plot performance across a grid of parameter values. You want a broad region where neighboring values give similar results. An isolated peak (0.4 at lookback 18, 1.9 at 20, 0.3 at 22) is _____.
- **Perturbation test.** Nudge every parameter by plus or minus 10 to 20%. If the result falls off a cliff, the rule is fragile.
- **Cost sensitivity.** Double your slippage assumption. A real edge survives; a fitted one does not.
- **Start-date sensitivity.** Shift the backtest start by one month in each direction.

## Working Rules for Trusting a Backtest

- Write your rules down *before* you look at results, and count your parameters honestly.
- Reserve a final holdout slice of data and touch it once, at the very end. Every peek makes it less useful.
- Demand at least 30 trades before believing any statistic, and prefer 100 or more.
- If a strategy needs a specific parameter value to work, it does not work.
- Report the bad numbers too: max drawdown, worst month, longest flat period. A backtest summary without drawdown is marketing.
- Be most suspicious of the result you like best.

## What to Carry Into the Checkpoint

You now have the full reading toolkit: equity curve shape, trade log distributions, a small metric dashboard, walk-forward splits, and parameter plateau checks. The one-sentence version: **a backtest is evidence only when part of the data was never used to build the strategy**.

Next up is the Module 2 checkpoint, where you will apply these checks to your own backtest and defend the result with numbers rather than a single Sharpe ratio.

Front: What is max drawdown?
Back: The largest peak-to-trough decline in the equity curve, the loss you would have had to sit through.

Front: Why is win rate alone a useless statistic?
Back: A high win rate with one huge loser can still lose money; expectancy combines hit rate with payoff size.

Front: How is expectancy computed?
Back: Win rate times average win, minus loss rate times average loss.

Front: What payoff ratio means
Back: Average winning trade divided by average losing trade.

Front: What does the Sharpe ratio hide?
Back: Fat tails and asymmetry: it treats upside and downside volatility the same and assumes roughly normal returns.

Front: Define overfitting in a backtest
Back: Rules and parameters fitted to noise in one historical sample, so performance fails out of sample.

Front: What is walk-forward testing?
Back: Optimizing on an in-sample window, testing once on the next out-of-sample window, then rolling both windows forward.

Front: Anchored versus rolling walk-forward
Back: Anchored grows the training window from the start; rolling keeps it a fixed length and drops old data.

Front: Parameter plateau versus spike
Back: A plateau is a broad region of similar performance; a spike is an isolated peak that is likely noise.

Front: What is the perturbation test?
Back: Nudging each parameter by plus or minus 10 to 20% and checking that performance does not collapse.

Front: Minimum trade count before trusting statistics
Back: About 30 trades as a floor, with 100 or more strongly preferred.

Front: Rough ceiling on parameters per trade
Back: About one parameter for every 30 to 50 trades in the sample.

Front: Red flag in profit concentration
Back: If a few trades supply more than roughly 30% of total profit, the result depends on luck.
A backtest that looks brilliant on the exact data used to choose its parameters is called _____, and its main symptom is a large gap between in-sample and _____ results.
Walk-forward testing rolls two windows forward: the _____ window picks the parameters, and the next _____ window scores them once.
A robust parameter sits on a broad _____ where nearby values all perform similarly, not on an isolated spike.
Before trusting any performance statistic, demand roughly _____ trades as a floor, and treat per-trade profit concentration as a warning sign.

## Habits That Keep a Backtest Honest

A few rules of thumb will save you from the most common beginner traps. None of them are exotic; they are just discipline applied consistently.

- Write the rules down before you look at results. If you change a rule after seeing the equity curve, you are fitting, not researching.
- Judge the strategy on out-of-sample numbers only. The in-sample period is for choosing parameters; it is not evidence.
- Prefer the middle of a plateau over the top of a spike. A slightly worse parameter that works across a broad region is worth more than a perfect one that only works once.
- Always compare against a benchmark. A strategy that returns 8% per year looks fine until you notice buy-and-hold returned 12% with lower drawdown.
- Sanity check the trade count. Under about 30 trades, statistics are anecdotes. A Sharpe ratio computed on 12 trades means almost nothing.
- Check profit concentration. If three trades supply most of the gains, the strategy is a lottery ticket, not an edge.
- Keep a research log. Note every change you make and why. When you have tried forty variants, the best one is partly the winner of a search, and your log tells you how much search happened.

> **Warning:** 
> The most expensive mistake in this lesson is testing many variants and reporting only the best. Each additional trial raises the odds that one of them looks great purely by chance. Track how many ideas you tried, and discount your best result accordingly.

## Quick Reference for Reading Your Own Results

| Signal you see | Likely meaning | What to do |
| --- | --- | --- |
| Smooth rise, tiny drawdown | Costs may be missing or too low | Re-check slippage and fee assumptions |
| Great in-sample, weak out-of-sample | Overfitting | Simplify the rule set, fewer parameters |
| One parameter dominates performance | Fragile fit | Run the perturbation test, look for a plateau |
| Fewer than 30 trades | Not enough evidence | Extend history or loosen the entry rule |
| Profit from a handful of trades | Luck concentration | Inspect those trades, test without them |

## Key Takeaways and What Comes Next

- An equity curve answers the "how much" question; the trade log answers the "how" question. Read both.
- Sharpe ratio and maximum drawdown are useful summaries, but they hide fat tails and clustering.
- A strategy that only works on the exact data used to tune it is overfit; walk-forward testing is your main defense.
- Flat parameters, enough trades, and diversified profit sources are the marks of a result worth trusting.
- In the next checkpoint you will apply all of this: you will read your own backtest output, sanity check the parameters, and defend whether your strategy has a real edge.

---

## Contents

Part of [📊 Quantitative Trading for Beginners: Build and Backtest a Strategy](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy.md)

- **📈 Market Data and Return Statistics**
  - [🧭 What a Quant Strategy Actually Needs](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/what-a-quant-strategy-needs.md)
  - [🗂️ Getting and Cleaning Price Data](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/getting-and-cleaning-price-data.md) (sign-in required)
  - [📉 Returns, Volatility, and Time Series Basics](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/returns-volatility-and-time-series-basics.md)
  - [✅ Checkpoint: Market Data and Statistics](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/checkpoint-market-data-and-statistics.md)
- **🧪 Strategy Design and Backtesting**
  - [📝 From Idea to Testable Trading Rules](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/from-idea-to-testable-trading-rules.md)
  - [🧮 Building the Backtest: Fills, Slippage, and Fees](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/building-backtest-fills-slippage-and-fees.md)
  - [🔍 Reading Results and Avoiding Overfitting](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/reading-results-and-avoiding-overfitting.md) — this lesson
  - [✅ Checkpoint: Strategy Design and Backtesting](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/checkpoint-strategy-design-and-backtesting.md)
- **🛡️ Risk Control and Capstone Build**
  - [⚖️ Position Sizing and Core Risk Metrics](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/position-sizing-and-core-risk-metrics.md)
  - [🏗️ Capstone: Build and Backtest Your Own Strategy](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/capstone-build-and-backtest-your-strategy.md) (premium content)
  - [✅ Checkpoint: Risk and Capstone](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/checkpoint-risk-and-capstone.md) (premium content)
- [Course cheatsheet](https://everwise.study/courses/quantitative-trading-for-beginners-build-and-backtest-a-strategy/cheatsheet.md) (premium content)