Prediction Market Trading Bot Backtesting: From Replay to Paper Trading
A bot is not validated because its signal predicted the right side. It is validated when the signal survives historical depth, costs, gaps and markets it has never seen.
A prediction market bot backtest should replay information in timestamp order, price every entry against the available side of the historical book, settle against the real outcome and preserve markets with missing coverage as missing. The next step is forward paper trading on live books. That two-stage process separates a historical idea from a strategy that can operate without look-ahead or midpoint-only fills.
The test ladder
| Stage | Question | Pass condition |
|---|---|---|
| Signal sanity | Does the rule use only information available then? | No settlement or future-row leakage |
| Midpoint replay | Is there directional value before execution? | Positive result under an explicitly optimistic fill |
| Depth replay | Does the edge survive spread and size? | Sized VWAP remains viable |
| Stress test | Does it survive gaps and parameter changes? | No single setting or market dominates |
| Paper trading | Does it work on unseen live markets? | Stable forward record with virtual cash |
Model fills before optimizing signals
A buy crosses the ask and a sell crosses the bid. If the requested size exceeds the first level, walk the ladder and compute VWAP; if the ladder cannot support the order, record the unfilled fraction. A midpoint fill can remain as a best-case ceiling, but it should never be confused with executable performance.
Fees, latency, queue position and market impact require separate assumptions. Keep each assumption visible so a result can be reproduced rather than hiding all execution error inside one unexplained slippage constant.
Use one rule across Polymarket and Kalshi
DepthFeed normalizes recorded Polymarket and Kalshi books into one research shape while preserving venue identifiers and source metadata. The Backtest Lab can run presets or a sandboxed custom rule over resolved markets with midpoint, fixed-slippage or book-depth fills.
A surviving rule can move to Paper Trading. DepthFeed evaluates supported rules server-side or accepts signals from a per-strategy webhook, prices virtual fills from the live book and records an equity curve, open positions and trades. Sports backtests are supported; forward sports paper execution is not available yet.
Bot validation checklist
- Freeze a market universe and time range before tuning.
- Keep outcome and settlement fields unavailable to the entry function.
- Compare midpoint, fixed-slippage and depth-aware fills.
- Report unfilled size, missing books and rejected trades.
- Run sensitivity tests around every threshold and holding rule.
- Forward-test the final rule before connecting real execution.
Key takeaways
- 01A bot backtest must evaluate information strictly in timestamp order.
- 02Depth-aware VWAP reveals edges that disappear when real size crosses the book.
- 03Missing data and unfilled size belong in the result, not in a synthetic fill.
- 04Polymarket and Kalshi can share rule logic when the book is normalized transparently.
- 05Paper trading supplies the forward evidence a historical backtest cannot.
Replay the signal against the historical ladder, then give the survivor a forward paper track record. Free Explorer tier, no card.
Open Backtest LabView pricing