Free tool
Is your backtest better than luck?
Try enough versions of a trading strategy and one of them will look good by chance alone. This works out how good "by chance" is, so you can see whether your best result clears it.
The luck line at a glance
| Versions tried | 1 year of data | 2 years | 5 years | 10 years |
|---|---|---|---|---|
| 10 | 1.54 | 1.09 | 0.69 | 0.49 |
| 100 | 2.51 | 1.77 | 1.12 | 0.79 |
| 1,000 | 3.24 | 2.29 | 1.45 | 1.03 |
| 10,000 | 3.85 | 2.72 | 1.72 | 1.22 |
The Sharpe ratio the best version reaches when none of them has any edge. A short test and many tries make a high number easy to get.
How it is worked out
A strategy with no edge still shows a Sharpe ratio above or below zero on any one stretch of history. Over one year that scatter is about 1; over four years it is about half that, because it shrinks with the square root of the length of the test.
Run many versions and you get many draws from that scatter, and you keep the best. The best of 10 draws sits about 1.5 above zero on average, the best of 100 about 2.5, the best of 1,000 about 3.2. Divide by the square root of the years and you have the luck line.
It assumes the versions are unrelated. Versions that are close copies of each other count as fewer tries, so the true line is a little lower. Treat it as a guide: a result under the line proves nothing, and a result above it has earned a test on data it has never seen.
The idea comes from Bailey, Borwein, López de Prado and Zhu (2014). In our own search about 19,000 rule sets were tried in two days and one passed the final test: what that taught us.
Free: the first chapter of our report
How a trading bot fools you. Luck is one of six ways a test flatters a trading bot. Read how our own test leaked, and the ten red flags to check on any bot. We email you the link, with one short desk summary a week after that. Unsubscribe in one click.