How the Lab tests a trading robot.
Six checks, run the same way every time, with the data, costs and settings written down so anyone can repeat them.
Six checks: what we test, how, and what passes.
- 01
Real ticks
Does the result survive every-tick data?
- How we test
- We replay the EA on real tick data from a live broker feed and record the tick quality MT5 reports. Before anything else, we try to reproduce the author’s or vendor’s own result.
- What passes
- The result reproduces on real ticks, with a matching trade count and a profit close to the advertised one.
- 02
Out-of-sample
Does it work on data it was not tuned on?
- How we test
- We run periods outside the showcase window: the months after it, the years before it, and one continuous run across both.
- What passes
- It stays profitable out-of-sample, over enough trades to mean something, with a drawdown close to the in-sample one.
- 03
Live and verified
Is real money behind it?
- How we test
- We look for a live signal verified by a third party, such as MQL5 Signals, and compare its drawdown and trade pattern with the backtest.
- What passes
- Several months of verified live trading that agrees with the tests. Screenshots never count.
- 04
Real costs
Are spread, commission and swap included?
- How we test
- We test on a raw-spread account model with its real commission, the spread recorded in the tick data, and swap switched on.
- What passes
- The edge survives the costs a real account pays.
- 05
Robustness
Is it one lucky setting?
- How we test
- We nudge each important input, move the start date by a few sessions, and run the EA on other markets where its logic allows. Then we count how many variants stay above zero.
- What passes
- Most variants stay profitable, and no single setting or start date carries the result.
- 06
Integrity
Does the code do what the listing says?
- How we test
- When the source is public we read it, looking for hard-coded dates, hidden martingale or grid logic, missing stop-losses and inputs that change nothing. We also check the lot size behind any advertised drawdown.
- What passes
- Every trade has a defined risk, and the advertised figures come from sizes a person would really trade.
Three results. One bar.
Every verdict ends in one of three words. Passing needs all six checks.
- Passed
All six checks pass. It is the Lab’s strongest result, and it is still not a promise about the future.
- Not proven
Nothing fails outright, but the evidence is too thin or too fragile to trade on. More data could move it either way.
- Rejected
At least one check fails on something that matters: no edge out-of-sample, a result that breaks when nudged, or a risk the listing does not show.
Test packs, right of reply and corrections.
Test packs
Every verdict names the version tested, the data, the broker model, the costs and every input. The settings files and reports are published with the final verdict, so anyone can repeat the test.
Right of reply
Before a verdict goes live we send it to the author or vendor and give them 14 days to respond. Their response is published next to the verdict, unedited.
Corrections
If you find an error, send it with the evidence. We re-test, and if we were wrong the verdict changes and the correction is logged on the page with its date.
About to buy an EA? Get it tested first.
A private Lab verdict on the six checks, before you pay the vendor.
Verdicts are the Lab’s opinion, based on reproducible tests of the version named. They are not investment advice. Trading on margin carries a high risk of losing money.