Evidence
4 Oct 2026, 21:33 UTCOver 13 paired rounds, the model no difference demonstrated (last rally order).
| Baseline | Race | Rounds | Model | Baseline | Gain | Verdict |
|---|---|---|---|---|---|---|
| Last rally order | rally | 13 | 8.25 | 8.33 | +0.08 | no difference demonstrated |
| Standings order | rally | 13 | 8.25 | 7.96 | -0.29 | no difference demonstrated |
Paired bootstrap on the round-by-round difference: 0.077, 95% CI [-1.140, 1.932] in positions gained. Positive means the model is closer to the real finishing order than last rally order is; the interval covers zero, so no difference has been demonstrated.
forward evaluation — each round was forecast before it ran and scored afterwards; not a backtest. Historical replays and the forward record are computed separately and never merged.
- Some comparisons straddle zero: a difference has not been demonstrated, in either direction.
- Metrics are only comparable within this series. A position error over this field size means nothing next to another series' number.
WRC · 2026
Model accuracy
How the WRC model’s leakage-safe pre-rally forecasts have scored against the actual results, over 13 completed rounds of 2026. Every number is scored finishers-only, using only data available before each rally.
Winner hit rate
23%
Podium hit rate
41%
Mean position error
8.25
NDCG@5
0.62
Model health at a glance
HealthyA self-check on the live model: whether recent forecasts are still landing as well as they should, and whether the numbers feeding the model have drifted from what it was tuned on.
Forecast quality
▼ 15%
lower error than the season benchmark
Rounds monitored
13
recent rounds in the rolling check
Input drift
2/5
model inputs that moved vs baseline
Input drift by feature
Some model inputs have drifted from their reference range — normal early in a season with a small sample of rounds so far. It is flagged here for transparency, but the rolling forecast-quality check shows the predictions themselves are still holding up.
Per-round accuracy
Podium-weighted rally accuracy per round. Tap a cell for the breakdown.
Per round
| Round | Winner | Podium hits | Mean error | NDCG@5 | Win Brier |
|---|---|---|---|---|---|
| R1 · Rallye Monte Carlo | miss | 2/3 | 6.231 | 0.96 | 0.0625 |
| R2 · Rally Sweden | miss | 1/3 | 6.583 | 0.87 | 0.0641 |
| R3 · Safari Rally Kenya | miss | 1/3 | 7.929 | 0.57 | 0.0623 |
| R4 · Croatia Rally | ✓ hit | 2/3 | 15.4 | -0.22 | 0.0426 |
| R5 · Rally Islas Canarias | miss | 1/3 | 8.091 | 0.86 | 0.0971 |
| R6 · Vodafone Rally de Portugal | miss | 0/3 | 8 | 0.69 | 0.1000 |
| R7 · FORUM8 Rally Japan | miss | 2/3 | 6 | 0.90 | 0.0781 |
| R8 · EKO Acropolis Rally Greece | miss | 1/3 | 6.308 | 0.87 | 0.0744 |
| R9 · Rally Estonia | miss | 1/3 | 9 | 0.24 | 0.0719 |
| R10 · Secto Rally Finland | ✓ hit | 1/3 | 9.083 | 0.92 | 0.0574 |
| R11 · Ueno Rally Del Paraguay | ✓ hit | 2/3 | 11.833 | 0.05 | 0.0496 |
| R12 · Rally Chile Bío Bío | miss | 1/3 | 6 | 0.44 | 0.0842 |
| R13 · Rally Italia Sardegna | miss | 1/3 | 6.786 | 0.95 | 0.0699 |
Win Brier scores the model’s win probabilities against who actually won — lower is sharper and better calibrated.
Vs the baselines
Does the forecast beat championship form?
A rally has no qualifying to lean on, so the fair test is whether the forecast is sharper than simply ordering crews by their championship standings. Over 13 completed rounds the model is sharper than championship form on the podium probability score, while championship form currently holds a slim edge on the win score. A last-rally momentum baseline is a touch sharper on the win score this season — shown here honestly rather than hidden.
| Measure | Our forecast | Championship form | Last rally |
|---|---|---|---|
| Win probability score(lower is sharper) | 0.863 | 0.861 | 0.813 |
| Podium probability score(lower is sharper) | 0.730 | 0.778 | 0.796 |
| Winner called(higher is better) | 23% | 15% | 31% |
The skill model alone scores 0.893 on the win score and 0.771 on podium — weaker than championship form on its own. Blending it with championship form keeps the combined forecast competitive with championship form.
Walk-forward validation
Model vs the championship-form baseline
Every completed round is re-forecast using only earlier rounds, then scored against a baseline that simply ranks crews by their championship standings. Gold marks the better side. Beating championship form on the ranking metrics is the bar the model has to clear.
Rally
13 rounds · model vs championship form| Metric | Model | Championship form |
|---|---|---|
| Mean position error | 8.25 | 7.96 |
| Top-5 ranking | 0.624 | 0.566 |
| Order agreement | 0.535 | 0.511 |
| Podium hits / round | 1.23 | 1.23 |
Probability calibration
How trustworthy the probabilities are
A well-calibrated model assigns probabilities that match how often things actually happen. WRC’s forecasts are tuned against the real classified rally results so a stated 30% podium chance means roughly 3-in-10 over the long run.
Training rounds
13
real completed rounds
Status
Calibrated on real WRC rally results (one classification per round).
Generated 10/13/2026, 4:24:15 PM
Calibration samples per market
Win
162
observations
Podium
162
observations
Top 6
162
observations
Top 10
162
observations
Each figure is how many prior crew-outcomes fed the calibrator for that market. More samples means a steadier probability estimate.
Model health
Win-market Brier trend
Lower is better · 13 rounds
Diagnostics
- ⚠ pPodium: PSI 0.289 (significant drift vs baseline)
- • pWin: PSI 0.187 (moderate drift vs baseline)
Feature drift and rolling-Brier are tracked round-to-round; a spike flags where the field behaved unlike the rounds the model learned from.