Why your backtest lies
We analysed 17 robots that shone on paper. None with a real edge survived the market's ticks. With data, not opinions.
The trap that blows up accounts
Almost every robot (EA) bought and sold comes with a spectacular backtest: high profit factor, flawless equity curve. The catch is that this backtest uses a synthetic tick model —prices generated by the simulator— not the reality of the market: real spreads, slippage and the microstructure of every tick.
The consequence: a robot can boast a Profit Factor of 1.68 in the simulator and lose money in real execution. This is not theory. We measured it, robot by robot, with the broker's real ticks.
The proof: simulator vs. real ticks
We put a family of 17 robots (variants of the same gold idea: order-block entry plus a trend filter) through a double judge —survival on real ticks vs. the synthetic backtest. In the simulator several looked like pure gold. On real ticks, the truth (representative sample):
| Robot | Sim PF | Real PF | Real net (1,000 $) |
|---|---|---|---|
| PHALANX | 1.68 | 0.74 | −878 $ |
| TITAN | 1.46 | 0.84 | −571 $ |
| IRON DOME | 1.40 | 0.89 | −395 $ |
| ADAPTIVE | 1.33 | 0.78 | −820 $ |
| VANGUARD | 1.25 | 0.74 | −196 $ |
| GHOST PROTOCOL 2 | 1.01 | 0.68 | −775 $ |
| V3.1 «Golden Phalanx» | 1.01 | 0.38 | −959 $ |
| V3 «Oracle of Delphi» | 0.95 | 0.50 | −857 $ |
The pattern is uniform and damning: the synthetic mirage collapses against real fills. And note: the two «favourites» with a Kill Zone time filter (Golden Phalanx and Oracle of Delphi) were among the worst in reality —PF 0.38 and 0.50, probability of ruin near 100%. A pretty filter won't save logic with no edge.
How to validate for real (the four filters almost nobody applies)
- Real-tick survival: the same robot on real ticks vs. synthetic. If the result changes, it was a mirage.
- Tail Monte Carlo: probability of ruin with the exact tail derived from the robot's logic, not an optimistic average.
- Walk-forward: does the edge hold on periods the robot never saw while being tuned?
- Plateau (sensitivity): perturb the parameters ±15%. If a single tweak breaks it, it was overfitting in disguise.
For contrast, a robot bred and filtered with this process: out-of-sample PF 1.31 · real-tick survival 99.8% · probability of ruin 0% · plateau 6/6 healthy variants. The difference with the table above isn't luck: it's the method.
Conclusion
Don't trust a pretty backtest. Before putting a real euro into a robot —yours or bought— the right question is: does it survive real ticks, its tail risk, and perturbing its parameters? If you haven't measured it, you don't know.