Before you start
Choose one competition, one target and one forecast time. A workable first project is regulation-time home, draw and away probabilities produced 24 hours before kickoff. Do not begin with several markets, live updates and player tracking at once: each adds a separate label, availability rule and validation problem.
You need fixture identifiers, kickoff times, final labels and at least one feature family available before the cutoff. StatsBomb Open Data provides inspectable match, lineup and event structures for selected competitions. Its public files are useful for learning, but coverage and licence terms must still be checked for the intended use.
Build the minimum model
- Write the target contract. Define the outcome, match period, exclusions and timezone.
- Create an availability table. Record both the event time and the time each input became available.
- Order fixtures by kickoff. Reserve the latest period as an untouched test set.
- Fit a baseline. Start with league frequencies or a transparent rating model.
- Add one feature family. For example, rolling goals or expected goals calculated from prior fixtures only.
- Fit one candidate model. Keep preprocessing, imputation and modelling in one versioned pipeline.
- Score later fixtures. Report probability scores, calibration, coverage and a relevant baseline.
- Freeze an evidence bundle. Save code version, data snapshot, parameters, predictions and evaluation output.
scikit-learn's common-pitfalls guidance explains why preprocessing must be learned from permitted training data. TimeSeriesSplit is one way to preserve order during development.
Minimal data contract
| Field | Purpose | Failure to prevent |
|---|---|---|
| fixture_id | Stable join key | Duplicate or mismatched matches |
| kickoff_utc | Ordering and cutoff | Local-time leakage |
| snapshot_utc | What was known when | Final-state data used too early |
| target | Outcome to estimate | Market-label mismatch |
| feature_version | Reproducible transformation | Silent definition changes |
| prediction | Probability vector | Class label without confidence |
| model_version | Reproduce output | Untraceable reruns |
Evaluate the output
For three-way outcomes, probabilities should sum to one after any declared normalization. Use log loss or another proper probability score, plus a reliability table that groups similar forecasts. scikit-learn's model-evaluation documentation describes task-appropriate metrics, while its calibration guide explains why a 0.60 forecast should be assessed across many comparable cases.
Illustrative release check
Suppose a baseline has test log loss 1.015 and the candidate has 1.002 on the same 900 later fixtures. The candidate is lower by 0.013 in that sample. Release still depends on calibration, uncertainty, missing-input behavior and stability by league or season; the difference does not prove a permanent advantage.
Common failure modes
- Computing rolling form after sorting incorrectly or including the current result.
- Filling historical lineups with final corrected records that were unavailable at the cutoff.
- Selecting features against the final test period.
- Comparing models on different fixture subsets.
- Reporting accuracy without the class rule, denominator or probability quality. scikit-learn's common-pitfalls guidance supports the leakage and pipeline controls, while its model-evaluation guide supports probability-aware reporting.
Verification checklist
- Rebuild one prediction from stored inputs and code.
- Confirm every rolling feature stops before the target fixture.
- Compare fixture IDs and exclusions across all models.
- Check that failed or missing predictions remain in coverage reporting.
- Run the untouched test only after the pipeline is locked.
Create a reproducibility handoff
A project is not complete when one notebook prints a score. Give another analyst a clean environment and ask them to reproduce one test prediction and the full report from the stored artifacts.
| Artifact | Acceptance check |
|---|---|
| Data manifest | File hashes, retrieval dates and licence are present |
| Split manifest | Every fixture appears in one declared role |
| Feature pipeline | Fit state comes only from permitted training rows |
| Model artifact | Parameters, runtime and code commit are recorded |
| Predictions | Fixture IDs, cutoffs and full probabilities are immutable |
| Evaluation | Baselines use the same fixtures and target |
scikit-learn's common-pitfalls guidance supports pipeline consistency and leakage controls. The handoff tests a further risk: undocumented local state that makes an apparently correct result impossible to rebuild.
Add production parity before release
Replay historical cutoff records through the production transformation and compare them byte-for-byte, or within a documented numerical tolerance, with the research pipeline. Test unknown teams, missing lineups, duplicate fixtures and provider corrections. Record whether the system withholds a prediction or uses a declared fallback.
Only then run the final later-period evaluation. A model can be statistically sound in research and still fail because deployment maps teams, timezones or missing values differently. scikit-learn's common-pitfalls guidance supports testing a consistent fitted pipeline rather than a separate research-only transformation.
Continue the workflow
Read machine learning for football prediction when choosing features, baselines and a supervised candidate for this project plan.
Continue learning
- Next guide: How to Evaluate an AI Football Prediction Service
- Related guide: Can AI Beat Bookmakers? What the Evidence Must Show
Assumptions and limitations
This is a project method, not a claim that a particular algorithm predicts football reliably or produces betting returns. The worked scores are illustrative. Dataset corrections, promoted teams, rule changes and deployment differences can invalidate an apparently sound backtest.
GambleAware
GamCare
Gambling Therapy
GAMSTOP