1. Home
  2. Findings
  3. Trading Bot Review: 10 Essential Checks to Avoid Mistakes

Trading Bot Review: 10 Essential Checks to Avoid Mistakes

A trading bot can look perfect in a seller’s backtest and still lose money the first month you run it. The equity curve is smooth, the win rate is high, and the screenshots look professional. None of that tells you whether the bot has an edge after real costs.

I am Theoneste NZAKIZWANIMANA, founder of TradeLab Research. We test bots independently for traders who want an honest answer before they buy or run one. This post is the checklist we use: ten checks, the red flags we look for in sellers’ backtests, and what our report contains. Client details stay private, so every example here comes from our own published research.

Everything below describes historical simulations, not investment advice. I am not recommending any bot, strategy or market, and nothing here promises a return.

Why a trading bot review needs an outside tester

The person selling a trading bot has every reason to show the best version of it. That does not have to mean fraud. It usually means the settings were tuned on the same years shown in the backtest, the costs were optimistic, and the losing periods were left out of the marketing.

The numbers on retail trading are hard. Studies cited by ESMA found that 74-89% of retail CFD accounts lost money, with average losses per client of EUR 1,600 to EUR 29,000. A trading bot does not change those odds on its own. It only trades a set of rules faster and without emotion, and if the rules have no edge, it loses faster too.

An outside test has one advantage: the tester has no stake in the answer. For us a “fail” is as useful a result as a “pass”, and that makes it easier to report what the data says.

There is a second reason. Once you have paid for a trading bot, or spent weeks building one, it is hard to look at it coldly. A fixed checklist, applied the same way every time, takes the decision away from hope and gives it back to the data.

The 10-point trading bot checklist

Here is the full list. Each check catches a different way a backtest can mislead you, and a single failed check can be enough to change the verdict.

Diagram of the 10 checks we run in every trading bot review, grouped into logic, costs and statistics
The 10 checks in our trading bot review: logic, costs and statistics.
  1. The trading bot’s own code decides every trade.
  2. The broker’s real costs are charged at every fill.
  3. Time is split: tuning years, validation years and a sealed lockbox.
  4. The result is compared with luck.
  5. Results are broken down by market and by period.
  6. The code is checked for bugs such as duplicate signals and missing timeouts.
  7. Stops that sit inside the spread are found and handled.
  8. The data is checked for months with zero recorded spread.
  9. Overnight financing is charged.
  10. A forward paper test comes before real money.

The sections below explain each check, why it matters, and what we found when we applied it to our own research.

Trading bot checks 1-3: logic, costs and time splits

1. The bot’s own code decides every trade

The first rule of our trading bot review is simple: the bot’s own logic decides every trade. We do not rewrite the strategy from a description and test our version. Clients send MT5 expert advisors, Python code or written rules, and we run that logic on years of broker data.

Everything else is checked from the outside. Our independent simulator takes the same entries and exits and recomputes the result, so if the two disagree, something is wrong. When we assessed a 39-strategy library this way, TradeLab’s own backtest engine agreed with our simulator, which is exactly what you want to see.

Fills happen at the next bar’s open, never at a price the bot could not have had. Stops and targets are followed minute by minute. If a stop and a target are both hit in the same minute, we count the stop first.

2. The broker’s real costs, fill by fill

A trading bot that trades often lives or dies by its costs. We use OANDA MetaTrader 5 1-minute data with the spread the broker actually recorded at each fill, not an average and not a fixed number typed into a tester.

Our measured medians since 2024 give you a sense of scale: EURUSD 0.6 pips (90th percentile 0.8), GBPUSD 0.9 pips, AUDUSD 0.8 pips, gold $0.18 and US100 1.8 points. For a trading bot with a tight stop, that spread can be a large share of the risk on every trade.

In our edge search, an Asian-session FX mean reversion rule made +0.04 to +0.11R per trade before costs and paid 0.14-0.22R per trade in spread. All 12 settings were negative after costs. We also test every pass with 1.5 × the spread, because spreads widen. Our trading costs post covers this in detail.

3. Time splits: tune, validate, then lock away

If a trading bot was tuned on 2017-2026 and then shown on 2017-2026, the backtest proves only that the tuning worked. We split time in order: SEARCH from 2017-06 to 2022-12, where tuning is allowed, VALIDATION for 2023-2024, run once on frozen settings, and a LOCKBOX from 2025 onwards that stays sealed.

The rules, settings grids and pass thresholds are written down and hashed before each run. Any correction is a dated amendment, never a silent edit. That stops anyone, including us, from moving the goalposts after seeing the result.

Here is why it matters. Our best candidate from one round, a trend-day continuation rule on US500, US100 and US30, made +0.13R per trade in SEARCH with t 2.84 on 985 trades. It was positive on all three markets and in 5 of 6 years. In 2023-2024 it made −0.022R per trade, and 49.5% of coin flips did as well.

Bar chart of a trading bot rule with t 2.84 in the tuning years and t -0.35 in the untouched 2023-2024 period
Strong in the tuning years, gone in the next two: why every trading bot needs a time split.

Our out-of-sample testing post explains the split in more depth.

Checks 4-5: luck and the breakdown

4. The luck test

A good result can still be luck. For each result we run 200 “coin-flip” versions: the same entry times, the same stop and target distances, but a random direction. For long-only rules we also run 200 random long entries on the same bars. A rule has to beat 95% of them.

The random-long test matters more than most people think. In a market that rose for years, almost any trading bot that buys will look clever. Our only library pass, a linear regression slope rule on BTCUSD H1, earned most of its gains from longs while Bitcoin rose about 8 times. Read more in our luck test post.

Luck also hides in the number of settings tried. If a seller tested hundreds of combinations and shows you the best one, it will look strong by chance alone. This is the multiple comparisons problem, and it is the reason our bar rises to t 2.5 when an idea has already been tested on the same data.

5. Results by market and by period

A total hides where the money came from. We break every result down by market and by year, because a trading bot that works on one market in one period is not the same as one with a general edge.

In our edge search, a Williams breakout rule looked fine in SEARCH. The breakdown showed its profit came entirely from BTC and ETH; without crypto it made about zero. In 2023-2024 the crypto effect vanished, with t-statistics of 1.02, 1.22 and −0.43. Our 20-day Donchian near miss had the same shape: Bitcoin and Ethereum gave 88% of its profit.

When one market carries the whole result, ask the seller why. If there is no reason beyond “it worked”, treat it as a warning.

Checks 6-8: bugs, stops and bad data

6. Bugs: duplicate signals, missing timeouts and wrong exits

The most common surprise in our independent bot testing is trading bot code that does not do what the written rules say. Typical examples are duplicate signals, where the trading bot opens the same trade twice, and missing timeouts, where a trade meant to close at a set time stays open.

Our own research had a bug like this, and it taught me to look hard. In round 1 of our edge search, a Williams breakout rule on indices “passed” 2023-2024 with t 2.24 on 868 trades. We then found its end-of-day exit sat at the broker’s daily break, so trades were held over the reopen and the weekend. The extra holding supplied about 120R of its 156R.

We voided that pass. The corrected rule failed. If the bug had been in a seller’s trading bot, the backtest would have looked excellent and the live account would have carried overnight and weekend risk that the rules never intended.

7. Stops that sit inside the spread

A trading bot sometimes places its stop so close that, at the moment of the fill, the stop sits inside the spread. The trade is effectively stopped out before it starts, and the result in R becomes meaningless because the risk is close to zero.

We found this in TradeLab’s own backtest engine during our library assessment: trades whose stop sat inside the spread produced results down to −10^11 R. Those trades are now skipped. If a backtest shows enormous single-trade results, look for this first.

8. Zero-spread months in the data

Data can lie too. The broker export we used recorded zero spread for whole months of BTC, about 12% of all BTC minutes. A trading bot tested on those months pays nothing to trade, which flatters any rule that trades often.

We replaced each zero with that month’s median spread. The lesson is simple: before you trust any backtest, look at the spread column. If it is flat, zero or missing for long stretches, the result is too good by an unknown amount.

Checks 9-10: financing and a forward paper test

9. Overnight financing

A trading bot that holds positions overnight pays financing every night. We charge it as a stated assumption: 3% a year on FX, 6% on metals, indices and oil, and 25% on crypto.

It decides results more often than people expect. Our only library pass, STR-017 on BTCUSD H1, made +0.32R per trade with t 3.1. With 25% a year crypto financing added, it no longer passed: on 2017-2023 its t-statistic was 1.94, under our bar of 2.0. Monthly trend following was positive before financing and negative after it in all 8 settings. Our swap fees post explains how to convert financing into R.

10. A forward paper test before real money

The last check is time itself. Even a result that passes every test above has only been seen on the past. A forward paper test runs the trading bot on new prices, with no changes allowed, before any money is at risk.

We hold our own near misses to this rule. A pre-FOMC drift rule on US indices made +0.040R per trade with t 2.38 on 191 trades, and two other rules came close to our bar. None of them counts as a pass. They are kept only for forward paper testing, with a fixed read-out date of 2027-11-30.

What these checks found in 39 strategies

In November 2026 we ran an independent assessment of a library of 39 registered strategies: classic rules such as breakouts, EMA crossovers, RSI and MACD signals, Bollinger reversion, Supertrend, Ichimoku, order blocks and liquidity sweeps. Each used default settings on EURUSD, GBPUSD, USDJPY and BTCUSD, on M15 and H1, from 2017 to 2026.

The result was 1 pass, 4 inconclusive and 34 fails. The one pass then lost its status once crypto financing was charged. The four inconclusive results were trend rules that were positive on BTCUSD H1 in 2017-2023 but not confirmed in 2024-2026.

Bar chart of 39 strategies: 1 pass, 4 inconclusive and 34 fail in our independent trading bot assessment
Our independent assessment of 39 strategies: 1 pass, 4 inconclusive, 34 fail.

Each strategy got its own report with suggested fixes. The most common were a regime filter (33 reports), a signal that points the wrong way (32), a session filter (29), dropping M15 (16), keeping more of the edge the spread eats (15) and reworking the entry itself (12).

“The signal points the wrong way” is worth a closer look. In 32 strategies, price often reversed right after entry, yet the opposite trade still lost after the spread. Flipping a losing trading bot does not make a winning one.

Horizontal bar chart of the most common findings in 39 independent strategy reports
The findings that came up most often across the 39 strategy reports.

We then tested the 25 most promising fixes on markets they had never seen: AUDUSD, GBPJPY and ETH. With a corrected bar of t 3.11 for testing 25 ideas at once, none of the 25 was confirmed. Good-sounding improvements are not proof either. If classic rules with default settings mostly fail this checklist, a new trading bot built on the same ideas has to clear the same bar before anyone should trust it.

Red flags in a seller’s trading bot backtest

When you review a trading bot that someone wants to sell you, these are the warning signs we look for first:

  • No costs, or a fixed tiny spread. Ask which broker’s spread was used, and whether it was recorded at each fill.
  • No overnight financing on a trading bot that holds trades for days or weeks.
  • One long backtest with no split. If the settings were tuned on the same years you are shown, the curve proves little.
  • Only the best market or period shown. Ask for every market and every year, including the bad ones.
  • Too-good fills. In our SMC tests, 45% of “limit” fills were at a better price than the level itself. Re-pricing them to the level turned the best setting from +0.133R per trade to about 0.000R.
  • Huge single trades. Check for stops inside the spread or broken data.
  • No trade count or a tiny one. We require at least 100 trades in validation, and 200 in the tuning years.
  • Growing position size after losses. A martingale) can make a curve smooth until the one losing run that ends the account.
  • No forward record. A trading bot sold only on a backtest has never been run on prices it had not already seen.
  • Hundreds of settings, one shown. That is overfitting until shown otherwise.

None of these flags proves that a trading bot is bad. Each one means the backtest does not yet answer the question you are paying to have answered.

What our trading bot report contains

Our bot testing service follows the checklist above. You send the trading bot (an MT5 expert advisor, Python code or written rules), plus the markets, timeframes and broker you plan to use. We run the bot’s own logic on years of that broker’s data with that broker’s costs.

The PDF report contains:

  • results in R after spread and financing, by market and by period;
  • the time split and how the bot did in each part;
  • the luck test: how the trading bot compares with 200 coin-flip runs;
  • bugs we found, such as trades the code takes that the written rules do not intend;
  • what to improve, with the reasons behind each suggestion;
  • sample trades drawn on the chart, so you can see the bot’s decisions for yourself.

The most common verdicts are the same ones we see in our own research: no edge after costs, trades the code takes that the rules never meant, and results that depend on one market or one period. Those are not pleasant answers. They are much cheaper to learn from a report than from a live account.

Key takeaways

  • A trading bot review should run the bot’s own code, then check every trade from the outside.
  • Charge the broker’s recorded spread at each fill and overnight financing on every night held.
  • Split time: tune on one period, validate once on the next, and keep a sealed lockbox.
  • Compare the result with 200 coin-flip runs, and with random longs for a long-only trading bot.
  • Break results down by market and year; one market carrying everything is a warning.
  • Look for bugs, stops inside the spread and zero-spread months before trusting any number.
  • Paper trade any trading bot forward before real money, even after a pass.

If you want to see the full method, read how we test. If you have a trading bot you plan to buy or run, we can test it independently on your broker’s data and send you the report: see bot testing.

Check a strategy before you risk money on it

Send us your bot or strategy and get an independent report that uses your broker's costs.

What to send us

  • The bot files: an MT5 expert advisor, Python code or written rules
  • The markets and timeframes it trades
  • Your broker, so the test uses its spreads and financing
Email your bot
Scroll to Top