Hello everybody,
I would like to talk in this post about the reliability of backtesting.
I have recently analysed an EA using a Walk Forward analysis over a 5 year period. I optimised using a 10 year insample with a 1 year outsample data, repeating the process 5 times. The aggregate results for the 5 years are quite good:
Before going live, I put the EA to work for 3 months on 10 demo accounts from 10 different brokers. From 23/8/21 to 26/11/2021 (weeks 35 to 48) and the results on the 10 accounts are very similar between them:
The surprise comes when I try to repeat these results in backtest with 3 different historical data (Metaquotes, Darwinex and Dukascopy). The results of the 3 backtests are also similar to each other, but do not match to those obtained during the forward test. Of the 13 weeks, only in 5 weeks, the results are the same (marked in yellow). I show you the previous table, but with the backtest data added to the table:
Graphically, the equity curves are totally different:
Despite the positive results obtained in these 13 weeks of forward testing, the disparity of results between Forward Test and Backtest invalidates the whole preliminary study and therefore I discard the EA.
I encourage all of you to perform this kind of analysis with your EAs, especially if you are trading with real money. I perform this analysis since summer 2019 and it allowed me to improve my results a lot.
Have a good trading!