Win probability models used in sports analytics (for example, ESPN's NBA win probability model) are generally well-calibrated, and their probabilities align reasonably well with observed outcomes, at least compared with typical human intuition.
NOT BSTOTAL BS
HARDLY BS — Verdict: Mostly True
Verified by Lenz ·
The Short Version
The available evidence indicates these win-probability models are broadly calibrated over many games and usually align reasonably well with actual outcomes. They are not perfect: some studies find underdog bias and phase-specific quirks, and they are not clearly superior to simple baseline models. The comparison to human intuition is supported more indirectly than directly, but the overall claim is largely accurate.
Caveats
Calibration is a long-run, aggregate property; a probability that looks absurd in one game does not by itself show the model is broken.
Evidence for being better than typical human intuition is mostly indirect, based on predictive performance rather than direct comparisons of human probability calibration.
These models can have systematic biases, including underestimating underdogs or reacting imperfectly at certain game stages.