Markets Prediction
Get Promotion Codes
Jump toBest PlatformsLive Odds

Calibration

Calibration measures how well a forecaster's stated probabilities match the actual frequency of outcomes. A forecaster who assigns 70% probability to events should see those events occur approximately 70% of the time.

Updated June 24, 2026Probability & Forecasting
TL;DR
A well-calibrated forecaster's confidence levels reliably match real-world outcomes. It is the foundation of trustworthy probabilistic prediction.

Key Points

✓Perfect calibration means events predicted at 60% occur 60% of the time, events at 80% occur 80% of the time, and so on across all probability levels.
✓Overconfidence is the most common calibration error: forecasters assign probabilities too close to 0 or 1 and are wrong more often than expected.
✓Calibration is measured visually with reliability diagrams and numerically via metrics like the Brier score and log score.
✓Prediction markets tend to be well-calibrated because mispriced contracts create profit opportunities that traders quickly arbitrage away.
✓Regular feedback and deliberate practice are the most reliable ways to improve personal calibration over time.

What Calibration Means in Practice

Calibration is the degree of agreement between a forecaster's stated probabilities and observed frequencies of outcomes. If you say an event has a 30% chance and you make 100 such predictions, good calibration means roughly 30 of those events actually occur. In prediction markets, contract prices are continuously driven toward calibrated values by competing traders seeking Edge. A price of 0.65 on a Binary Market ideally reflects a true 65% probability of the event resolving YES. Poor calibration in a market signals a potential Mispricing that informed traders can exploit for profit.

How Calibration Is Measured and Improved

The most common numerical tools for assessing calibration are the Brier Score and Log Score, both of which penalize forecasters more heavily for confidently wrong predictions than for modest misses. Reliability diagrams plot predicted probabilities against actual outcome rates; a perfectly calibrated forecaster produces a straight diagonal line. Research from the Good Judgment Project shows that top crowd forecasters achieve significantly better calibration than domain experts by updating incrementally on new evidence using prior probabilities and adjusting to form posterior probabilities. Regular scoring feedback is the single most effective tool for training better calibration.

Related Terms

Brier Score
The Brier score is a numerical measure of the accuracy of probabilistic predictions, calculated as the mean squared difference between a forecast probability and the actual binary outcome. Lower scores indicate better forecasts, with 0 being perfect and 1 being maximally wrong.
Log Score
The log score is a strictly proper scoring rule that evaluates probabilistic forecasts by taking the negative logarithm of the probability assigned to the outcome that actually occurred. Higher (less negative) scores indicate better forecasts.
Prior Probability
Prior probability is a forecaster's initial estimate of an event's likelihood before incorporating new evidence, typically anchored to historical base rates or expert knowledge. It is the starting point for Bayesian probability updating.
Wisdom of the Crowd
Wisdom of the crowd is the phenomenon where the aggregated judgments of a large, diverse group of independent individuals produce more accurate estimates than most or all of the individual members. It is a core theoretical justification for prediction markets.
Mispricing
A mispricing occurs when a prediction market contract price deviates materially from the true probability of the event, creating an opportunity for traders with accurate probability estimates to capture positive [[expected-value]].

Find the best odds on every market

Compare live prices across Kalshi, Polymarket, and more — spot arbitrage and trade the sharpest line on any event.

Compare Markets