glossary
The most common way a profitable backtest becomes a losing bot. The fix is not a better optimiser; it is counting how many things you tried and testing on data the strategy never saw.
Backtest overfitting is the situation where a trading strategy's rules or parameters were adjusted to historical data until the backtest looks good, so the result reflects noise in that data rather than a repeatable edge. An overfit strategy typically performs far worse on new data than in its backtest. It is detected by counting the number of variants tried (deflated Sharpe ratio) and by testing on held-out data (walk-forward, holdout).
Official Binance broker · Non-custodial · Live only when you confirm · Updated
You test a 20-bar breakout, then 30, then 50, then add a filter, then remove a coin. Each try is a draw from the same data. After enough draws something looks good by chance, and the best of twenty worthless variants can show a Sharpe above 1.
A Sharpe that falls sharply when a parameter moves one step; most of the profit from a few trades; a perfect in-sample curve with no flat periods; a strategy that only works on the coin and dates it was built on.
Every run returns an overfit verdict based on how many variants of the idea were tried in the last 30 days, using the deflated Sharpe ratio. Studies group every variant of one idea so the final verdict counts all of them, and a walk-forward or holdout run tests on unseen data.
faq
No. A single backtest is in-sample by definition. OptAlgo treats it as a screen and asks for a walk-forward or holdout before deployment is advised.
It helps, but counting variants and testing out of sample fix it. A six-year backtest tuned over fifty variants is still overfit.