Textbook stop
79%One lap before he pitted on worn mediums, for the reasons a strategist would give: tyre age and pace drop-off.
Explainable AI · Formula 1 · 2023–2026
Can you trust an AI race strategist's reasons?
I trained a model to predict F1 pit stops from public timing data, explained it two different ways, and tested whether those explanations hold up. They were honest about the model and caught real mistakes. On any single lap, though, they often disagreed.
The setup
One question, asked on every lap
Will this driver pit in the next three laps? The model only sees what the pit wall knows at that moment.
86races, 2023–2026
92,091driver-laps
~9%of laps lead to a stop
18features per lap
Two models, explained two ways. XGBoost is a black box, explained afterward with SHAP. An Explainable Boosting Machine (EBM) is a glass box: its prediction is literally a sum of one score per factor, so its explanation is exact.
Tested on races it never saw. Trained on 2023–24, tested on 2025, then stress-tested on 2026, when the cars, engines and tyres all changed.
No peeking at the future. No race appears in both training and testing, in-laps are dropped, and the safety-car feature uses only each driver's own next lap.
Tyre age & compoundPace drop-offGaps ahead & behindLaps leftRivals pittingCar ahead pittedSafety car on next lapTwo compounds usedTrack-temp changePosition, stint, rain
Does it predict pit stops?
PR-AUC on the unseen 2025 season (higher is better; random guessing scores the pit rate). It warns before 69% of real stops, but most of its alerts are false alarms. Useful but often wrong, which is exactly when its reasons need checking.
Results
Four tests of the reasons
Both methods say which factors pushed each prediction toward or away from a stop. Here is whether those reasons can be trusted.
Agreement · do the two methods match?
0.95 / 0.50
Across the season they rank the factors almost identically. On a single lap, they usually share only two of their top three reasons.
Rank correlation of average importance · top-3 overlap per lap (Jaccard)
Stability · same story after retraining?
0.995
Retrain XGBoost five times with different random seeds and SHAP tells the same story every time.
Mean pairwise rank correlation across 5 seeds
Faithfulness · are the reasons real?
39→18%
Remove SHAP's top reason and the predicted pit chance halves. Remove a random factor and it barely moves (36%).
Deletion test on 1,285 laps where XGBoost predicted a stop
2026 rules · still the same logic?
0.99
New cars, engines and tyres, and the model reasons about pit stops almost exactly as before.
Rank correlation of SHAP importance, 2025 vs. 2026Remove the reasons, watch the prediction fall
Each step replaces one more of a lap's top-ranked factors with a value from a random other lap. A faithful ranking makes the line drop fast.
What matters most, per method
Average influence on the pit prediction (log-odds) on the 2025 season. Same leaders, different weights.
Debugging
What the explanations caught
The most useful moments came when an explanation didn't match how F1 strategy actually works. Each one led to a change, re-tested on races the model had never seen.
A shortcut
Track temperature ranked #3
Temperature varies about 9°C between races but only 1.3°C within one, so the model was using it to recognize which race it was. SHAP was faithful; the model's reason was bad. Faithful isn't the same as meaningful.
A blind spot
Safety cars barely mattered
Yet 27% of stops happen the lap a safety car or VSC appears. Adding "safety car on the driver's next lap" raised 2025 PR-AUC from 0.319 to 0.330.
A fix that failed
Temperature change instead
It won on the 2024 development data and lost on 2025. Every explanation-driven fix has to be re-tested.
Each fix, tested on 2025
Mean PR-AUC over 5 seeds; whiskers show ±1 standard deviation. Dropping temperature entirely did best, a reminder that the "fix" I kept was chosen before seeing 2025.
Case studies
Three real calls, three outcomes
Right for the right reasons, saved by debugging, and confidently wrong. Open any of them in the replay to see the reasons lap by lap.
Safety-car call
40%Russell pitted under a VSC on fresh tyres. The safety-car feature added after debugging is what flagged it.
The gamble
85%Sure he would stop, but he ran to lap 46 on tyres that were still fast. The model learned what teams usually do.
Takeaway
The explanations were honest about the model and helped fix it. But on a single lap two methods often disagree, and the model is often wrong. Use explanations to debug a model, not to justify a single call.
Limits
- It learns what teams did, not what was optimal, so it copies their habits, good and bad.
- Public timing data can't show tyre wear, team radio, forecasts or each team's plan.
- The deletion test creates unrealistic laps; it is compared with a random baseline for that reason.
- Correlated factors (race progress and laps left) can split credit differently between methods.
Next steps
- Run it live during races from F1's timing feed, updating predictions and reasons each lap.
- Add data the pit wall actually uses, such as weather forecasts and tyre-wear estimates.
- A user study: do these explanations help analysts predict team calls?