Run & evaluate

The run library

Runs are not throwaway. Every preview, test suite and backtest lands in the Runs section with its exact code version, parameters and data slice — searchable, comparable, exportable. This page covers organizing the library and, importantly, how to read the robustness views: regimes, stress, and price-path.

Finding and organizing runs

• Search and filters. Free-text search covers titles, notes and run ids — paste a whole run id, or any piece of it four characters or longer. Filter by kind (preview / backtest / test / optimize / significance), status (queued, running, succeeded, failed, cancelled, timed out…), verdict (GO / NO-GO), grading basis (legacy / realistic host — see the verdict basis), strategy, and date range. Every graded row wears its basis tag, and a page that holds runs on both bases says so above the list.
• Pin (★) what matters. Pinned runs move to the top of the library and are exempt from retention cleanup and purge — pinning is how you say “this is evidence, keep it”. Each pin can carry a private note (“Why does this run matter?”) displayed as Why pinned: …
• Name and annotate. Name a run when you launch it (the optional Name box) or rename it later wherever its name is shown — double-click its tab (or press F2 on it), the pencil on its run-history row, its Data Lens row, or here in the library. Project owners and editors can rename; viewers can't. The name shows wherever the run is listed or picked; the run history shows its strategy beside it. A run an AI agent started keeps its [AI] mark through any rename. Use Notes for observations shared with your project.
• Whole run ids. Every run id is shown in full (all 36 characters), with a Copy ID button beside it where space is short — the run, results and preview pages, each run-history row, and under the run pickers that can't fit it.
• Purge deletes a run's stored results (with confirmation). Pinned runs can't be purged until unpinned.

When a run fails

A failed run tells you three things, and it is honest about the third when it does not know.

• What broke, with the file and line. When your strategy raises, the error card names the failing file and line (strategy.py:12) and the engine's real traceback expands underneath it — not an exit code. Paths are shown relative to your workspace; our container paths never appear.
• Whose problem it is. If the failure was ours — a platform fault, a worker that died — the card says so plainly, tells you it was recorded for us to investigate, and does not offer to “fix” your code, because there is nothing in your code to fix. If the run stopped because the market data it needs is not in the catalog, it points you at the quoted download request instead.
• An honest “we do not know”. Many failures cannot be attributed reliably — a wall-clock timeout, for instance, is as often a runaway loop in a strategy as a slow platform. In those cases we make no claim at all and leave every option open, rather than guessing and sending you to debug code that was fine.

Fix with AI appears on failures that are yours to fix, and opens the AI edit lane pre-filled with the real error. Any proposal still comes back for your review.

When a backtest needs more memory than one run can have

Before a backtest is queued, the platform checks how much memory earlier backtests of the same kind on the same instrument actually used. When this one would need more memory than the platform can give one run right now, the launch is refused straight away — nothing is queued — and the launch dialog says so in plain words.

Changing the dates would not help: the memory a backtest needs does not depend on how many sessions it covers. The one exception is an instrument the platform has not measured yet, after a longer backtest on it ran out of memory: then the dialog says the backtest may need more memory than one run can have (no measurement stands behind that refusal yet) and suggests running a shorter window first, so the platform can measure it.

Press Report this in the refusal box. It opens the platform's feedback form already filled in as a bug report — the market (or markets, for a run that reads several), the window, the date and your project as the reference — so the platform team knows exactly what to make room for. Read it, add anything you like, and send it; you land on your report's page, where its status changes appear. Reports are posted on the feedback board, which every signed-in user can read.

Comparing runs

Two comparison tools cover two different questions:

Compare selected (2–3 runs) — “what changed?”

Select two or three runs and press Compare selected. You get: per-run summary cards; an equity overlay (one color per run); an aligned stats table with deltas against the baseline run; a config diff (exactly which settings differ); and the gates side by side — decision, grade and reasons per run. The first selected run is the baseline; all deltas read “baseline → run”. The URL is shareable and bookmarkable.

If the selected runs were made by different engine versions, an Engine mismatch chip under the heading says so: their fills and statistics are not like-for-like, so read each run on its own. Every run records the engine version and commit it ran on (the provenance stamp beside the chart and the report shows both). The same chip row names three more ways two runs can fail to be like for like: graded on different bases (legacy vs realistic host) or in different units (R vs dollars — one run had trades without a stop), on different exchange profiles, or on different session models; the gates table carries a basis column per run, and a gate graded in dollars shows its dollar figure under its own name.

Comparison table (any number) — “what's the landscape?”

The Comparison table shows one row per run and instrument, with sortable columns — trades, expectancy (R), profit factor, win rate, net P&L, Sharpe, max drawdown — plus the verdict and the grading basis. Click a column to sort; expand rows to overlay their equity curves. Use it to survey a whole family of experiments at once.

Engine-native runs — what the run page shows

A run of an engine-native strategy (see writing strategies; previews, backtests, studies and screens launch from the product) shows the simulated exchange's own records: the orders with their final status, one row per fill event, the positions with when they opened and closed, every order's event trail, and the trades — one row per closed cycle with its exit reason (force_flat when the platform closed it at the session's end), its commissions, and the largest favourable and adverse excursion filled in from the run's own prices. The entry funnel's eight counts per session — entries and how many were accepted, filled, cancelled, expired, denied or rejected — ride the run's order-funnel table and its manifest until the fills tab reads them (a later bundle). Each traded session gets a chart whose markers sit on the candle that was open at the fill — an entry, an exit named with its reason, a dot per partial fill — and a resting order draws as a level over the bars it rested. Clicking a trade in the Trades tab opens that session's chart showing only that trade — its entry, exit, partial-fill dots and entry, stop, target and exit lines — at every timeframe and as you scroll; tick Show other trades above the chart to see every trade again (the choice is remembered). A trade held into a later session shows its entry on the first day and its exit on the later one, and the window loads the sessions in between by itself; the chart runs from 5s to 1h candles (see timeframes). A run keeps a chart for its newest traded sessions up to a per-run limit an administrator sets (2,000 by default; it can also be set to keep every session). A trade on an older session says so instead of showing a chart — how many sessions the run traded, the limit, how many older sessions have no chart and that session's date — and offers Build this session's chart: the platform rebuilds that one chart from the run's own results and the data it read, and it appears in the panel when done. If that session's data is no longer on the platform as the run read it, it says so rather than draw a different chart. When there is not enough memory free for the build just then, the panel says it is waiting for memory, with how long it has waited, how many other charts are waiting ahead of it and when it gives up (there is no finish estimate: how long the work ahead of it takes is not known in advance), and the chart starts on its own as soon as there is room — or, after 30 minutes unless an administrator changed the limit, it says it could not start and asks you to try again later. A session the run never played (a market had no data for it) has no chart to build, and the panel says so. The provenance tab's run options table shows every option that applied with its three layers side by side — Platform default, From your strategy (your declared RUN_OPTIONS) and Your choice (the launch dialog) — and the value In effect with a chip naming the layer it came from (an option left at the platform default is shown once). The run's Instrument line carries the same chip and says where the market came from, e.g. “chosen on this launch; your strategy's own INSTRUMENT line says ES.v.0”. A market the dialog picked for you, because your strategy names none, reads Your choice there: the run records it as a launch choice. Since October 2026 such a run also stores its performance report: the performance tearsheet (the statistics page drawn in the platform's theme with the platform's own charts — the R distribution, the order funnel, MAE / MFE, PnL by hour and by session, the strategy against the random-entry control, the cost grid), the price bars with fills drawn during the run, a trade record (one entry per filled trade: its fills, planned levels, result and excursion — the trades tab renders it) and one per-trade page per filled trade over the run's own 5-second bars — 15 minutes before the entry and after the exit unless you set your own window under Settings → Results (an administrator can set it for everyone instead); the same window frames the Trades tab's chart when you select a trade, and each run keeps the window it was made with. A per-trade page reaches further back when a level your strategy plotted before the window still stands at its edge, so the setup's formation is on the page (never more than a day before the entry; past that, the page says the level is drawn from its left edge); its entry and exit markers name their fill price when you hover them; and it draws your session's levels plus only the levels you bound to that trade (scope="trade" — see the chart guide). A run that read several markets (since October 2026) names them on its page — each market with its role (the primary, usually the one the orders go on, or one read beside it) and where it came from — then the markets that received an order and how many sessions it ran on. Charts are built for markets with filled trades: the traded list names markets with orders, so a market that ordered but never filled is listed and has no chart. The Trades tab gets a market filter; each chart draws the other markets on their own panes under the price (each at its own price scale) with a Swap button that puts one on top — the bar-by-bar Replay too, revealing them on the same clock as the traded market (where the traded market had no bar for a stretch, the other market's bar for it appears once the traded market's bar that covers it has closed) — and each per-trade page draws them on a pane below the traded market with the same Swap and a Hide / show other markets button. If you would rather not see them, untick Show other markets — in the chart's header, on the per-trade page viewer, or under Settings → Results; it is one remembered choice for every run (a downloaded per-trade page still opens with them shown). What a run does not carry is said on the page rather than hidden — a report step that did not render is named on the run page, and the tables and charts are complete evidence on their own.

Exporting

From a run's results you can export CSV (stats), CSV (trades), JSON or XLSX; comparison pages export their tables as CSV. You can also generate a self-contained HTML report bundle (stats, gates verdict, equity/drawdown charts, provenance) from Share & report, and mint expiring read-only share links for teammates — links are revocable, every access is logged, and viewers still need to sign in.

A finished backtest's performance tearsheet lives on the run's Overview (the receipt) and at the bottom of its Equity tab: View renders the page in-app, Open opens it in a new tab and Download saves a self-contained file. The receipt says what the page was rendered with and when, and carries Re-make the report: the page is drawn again from the run's stored results with the platform's current theme, chart list and benchmark line — no re-run, no engine time. A run made before the report's inputs were stored says so and offers a re-run instead.

Every chart on the tearsheet gets a full-height row, and the first row is as tall as its two tables — Run information and Performance statistics — need, so neither scrolls inside its box. Pages made before November 9, 2026 used a tighter automatic layout, with small charts and tables that had to be scrolled: Re-make the report draws such a page again with the current layout from its stored results, with no re-run. In the app the tearsheet's frame grows to the page's full height, so you scroll the results page itself rather than a small box inside it; Expand still opens it full-window.

A backtest's charts and report are made right after its engine run. When a large run's charts and report cannot be made inside the run, the run still finishes with its results, trades and tables, and the platform builds the missing pieces — the trade record, the session charts, the benchmark comparison, the performance report and the per-trade pages — right after it, as a continuation of the same run: the run page says they are being built and shows them when they land (a notification tells you too). If they could not be built, the run page says why and offers Continue: build the missing report, which builds them again from the run's stored results and the market data it read — never a new run, no engine time. When the market data the run read is no longer on the platform as it was, the charts can never be drawn from it: the page says so, offers no Continue, and a new run of the strategy draws its charts from the data on the platform now.

Individual charts have their own controls, separate from these data exports: any results chart can Expand to fill the pane — it reflows responsively as the window resizes — and each one downloads as a self-contained file you can drop into a report or share as-is.

The app's own metrics charts — equity curves, distributions, the sensitivity heatmap, the admin metrics grid — carry a separate full-screen control: a small expand button in the chart's top-right corner, which appears when you hover the chart and when you reach it with the Tab key. Escape, the close button, or a click outside brings the page back, and nothing about the chart is lost. Charts too short to host the button without covering their own data skip it — the community leaderboard's sparkline has none anywhere, and the small stat-card sparklines and the progress score gauge have none on a phone or tablet, where the button is a permanently painted 44 px target.

The equity tab and its benchmark

A strategy that made money isn't automatically a strategy that earned it. The results viewer's Equity tab opens with the run's equity results, drawn from the run's own stored tables, and ends with its stored performance tearsheet (which counts percent of the account balance by calendar day, compounded, so one of its months can differ from the same month above — the page says so under its heading):

• The numbers first. A strip of the run's stored results — net P&L, the max drawdown in dollars and in percent, the longest drawdown in trading days, the Sharpe ratio with whether the sample is large enough to trust it, trades and trading days — exactly the values the Overview's summary shows (all markets combined) — then the recovery factor (the net profit divided by that max drawdown) and how far below its high the run ended under water, and for how many sessions.
• One switch: Daily, Weekly, Monthly, Yearly. Under the strip, the period you pick shows the typical, best and worst period, the share of green periods and the average green and red one. A week is the exchange sessions from its Monday, named by that Monday’s date. These figures count complete periods only: a week, month or year the run’s window starts or ends inside is partial, left out, and named (“2 partial weeks left out of these figures”). A figure that needs more complete periods than the run has shows a dash and says how many it needs. The choice rides the page’s address, so a link opens the same view; Settings → Results sets the period the tab opens on.
• The equity curve in dollars — the running total of the run's daily net P&L over every trading day in the window, flat days included (or, when the run stored its starting capital, the account balance with its high-water mark) — then the underwater (drawdown) curve, the five deepest drawdowns (how deep, from which high to which low, when each recovered or that it is still under water) and the bars of the chosen period, a partial period drawn as an outline. The underwater curve is measured at the close of each session while the strip's max drawdown is measured trade by trade, so the two can differ: a drop that recovers within one session shows only in the stored figure. The page says so under the chart. Every number is computed by the platform from the run's stored results — a percentage is on the run's fixed starting capital, never compounded — and agents read the same numbers.
• A benchmark overlay, on by default, drawn as a dashed line on the equity curve and on the underwater curve, so “did it beat doing nothing?” is always in frame — and a strategy that “wins” while drawing down worse than buy & hold is visible, not hidden.
• Buy & hold (the default) — the run's own instrument(s) over the same window, priced from the same bars the run consumed. Multi-symbol runs get an equal-weight basket of the run's own symbols, with the legs named in the label. The series is cost- and capital-adjusted: your run's own starting capital split equally across the legs, the engine's commission rules applied per leg, and a round trip charged at each quarterly roll — so the comparison is same-capital, after costs. A run that stored only the raw series falls back to it (one frictionless contract per leg), and the info note says so plainly instead of pretending the costs were modeled.
• Flat (cash) — the zero line: “did it beat doing nothing after costs?”
• Your choice sticks. The Benchmark overlay checkbox turns it off, and the on/off state plus your chosen benchmark persist on the device where you set them.
• The verdict keeps the comparison. The audit computes Benchmark excess (strategy net minus benchmark net) and Benchmark correlation (daily P&L) as stored verdict inputs, each a card on the Audit tab: a strategy that failed to beat its own buy-&-hold basket after costs, or whose daily returns are mostly the market's, shows it in the verdict, not just the chart. The two cards link back to this tab.
• The tearsheet comes last. Its equity curve carries the benchmark line the platform chose for the report (buy & hold at cost parity, or none — an administrator's setting); its drawdown chart is your strategy's alone, and the page says so under it (the benchmark's own drawdown is in the underwater curve above).

The two header numbers an older version of these charts showed (excess net and correlation, computed in the browser) are not drawn — they live on as the Audit tab's stored cards. The benchmark series is a stored run artifact — never recomputed, never a data purchase; the oldest runs, which stored only an end-of-window benchmark figure, show it as a marker line with a note. A run that stored no daily results says why where the charts would be. A run that stored no tearsheet says so at the bottom of the tab (the charts above still draw from the run's tables): one that kept what the page is drawn from can re-make it from the Overview's receipt (no re-run); running an engine strategy again stores the page.

"Explain this run" — a plain-English readback

Every succeeded backtest's Overview tab ends with an Explain this run panel: click Explain and the AI reads back the stored record — the headline numbers, the go/no-go decision and its reasons, every audit check by name, and the reality verdict when one exists — in plain English. It cites what the platform concluded and never softens it: a flagged verdict stays the headline. Nothing is recomputed, nothing is stored, and each click is one metered AI call under your quota (the panel says so before you spend it). The Audit tab remains the evidence; this is the narration.

The robustness views — and how to read each one

A single graded number can hide a fragile strategy. The results viewer's robustness tabs each attack the result from a different direction. None of them change the stored grade or verdict; they exist to tell you how the result might be wrong.

Regimes — “when does this strategy work?”

The Regimes tab slices the run's own trades by the kind of day and moment they happened in: session type (trend vs range), volatility versus recent sessions, close versus session VWAP, prior-day range position, opening-range expansion, contract-roll windows, liquidity, order-flow skew, time of day, day of week. Each slice shows trades, expectancy, profit factor, win rate, drawdown and net.

How to read it:

• Tight dispersion across slices — expectancy roughly similar everywhere — suggests a robust edge that doesn't depend on catching one kind of day.
• Wide dispersion — all the profit in one slice — means the “edge” is really a bet on that regime. That's not disqualifying, but you should know it, and the regime had better be identifiable in advance.
• Slices with too few trades are withheld (“insufficient sample — statistics withheld”) rather than shown as noise. Slices are descriptive research cuts, not promises.

Stress — “how does this die?”

The Stress tab replays the run's own stored trades through a battery of harsher what-ifs: extra slippage and commissions, randomly missed fills, resampled trade orderings, best winners removed. Press Run stress tests; each scenario gets a verdict — survives, fails, or informational — and findings are grouped into named warnings such as Cost sensitivity, Overfitting risk, Profit concentration and Fill realism.

How to read it:

• The baseline column is what actually happened; every stressed column is a labeled hypothesis, not a re-run of reality.
• A strategy that dies the moment costs rise a notch has no margin for the real world's frictions.
• If removing the top handful of winners flips the result negative, the “edge” was a few lucky trades (Profit concentration).
• A passing battery is a robustness argument — never a forecast.

Price-path robustness — “was it tuned to this exact tape?”

The price-path check re-simulates the strategy on altered copies of the price window itself — jittered, resampled, block-shuffled — usually a few hundred of them. If the strategy's logic captures something real about how the market moves, it should perform comparably on paths that preserve the market's character but not its exact sequence.

How to read it:

• The verdict is either holds up or likely overfit. The table shows your original path's result against the distribution across altered paths (best 5%, median, worst 5%, and a p-value).
• The key tell: if the original result beats even the best 5% of altered paths, the result is consistent with being tuned to that exact path — the app flags this rather than hiding it.
• Since 2026-10-22 every altered path is a real run of your strategy on the run's own simulated exchange (engine-native strategies and built-in templates; an old-form plugin's run is refused by name) — still directional evidence about overfitting, never a restatement of the run's own results. How many paths a check runs is a platform setting, and a check too large for the platform's budget is refused before it starts, with the reason.