Model calibration: do confidence scores match real accuracy
Model calibration is the degree to which predicted probabilities reflect true outcomes. Many models, including deep neural networks, produce confidence scores that do not match reality. Calibration curves and post-processing methods such as temperature scaling can help diagnose and correct this mismatch.
What does it mean for a model to be well calibrated?
A well-calibrated classifier is one whose predicted probabilities can be interpreted directly as confidence levels. For instance, if a binary classifier assigns a probability near 0.8 to a set of samples, then roughly 80 percent of those samples should actually belong to the positive class. This property is described in the scikit-learn probability calibration documentation.
Calibration matters because confidence scores are used for decision making, such as filtering, ranking, or risk assessment. If a model's confidence does not match its actual accuracy, the error can propagate into downstream tasks. Some algorithms, such as logistic regression, tend to produce well calibrated probabilities when the model is well specified, while others like naive Bayes or random forests show systematic biases.
Why are modern neural networks often poorly calibrated?
Research by Guo and colleagues found that modern neural networks are less calibrated than models from a decade earlier. The study On Calibration of Modern Neural Networks points to model depth, width, weight decay, and batch normalization as important factors affecting calibration.
Deep models tend to produce overconfident predictions, especially when trained with cross-entropy loss. The paper also shows that simple post-processing methods can substantially improve calibration without retraining.
Calibration in language models
Language models also face calibration challenges. A study by Kadavath and colleagues, Language Models (Mostly) Know What They Know, found that larger models are well calibrated on multiple choice and true/false questions when the answers are formatted appropriately.
The models can also evaluate the validity of their own generated answers by predicting a probability of correctness, which improves with scale and when the model considers multiple candidate answers.
Beyond accuracy: calibration as a separate evaluation metric
Accuracy is the number of correct predictions divided by total predictions, as defined in the Google for Developers Machine Learning Glossary. However, accuracy does not reflect how well the confidence scores match the actual outcomes. A model can be highly accurate yet poorly calibrated, especially on imbalanced datasets.
Holistic evaluation frameworks treat calibration as a distinct metric. The HELM evaluation of language models measures seven metrics including accuracy, calibration, fairness, and others, so that tradeoffs between them are visible. This approach ensures that calibration does not get ignored when models are compared.
| Method | Type | Description |
|---|---|---|
| Platt scaling | Parametric | Fits a logistic function to transform scores into probabilities |
| Isotonic regression | Non-parametric | Fits a piecewise constant, monotonic function without a fixed shape |
| Temperature scaling | Single-parameter | Divides the logits by a learned temperature, a variant of Platt scaling |
Frequently asked questions
What does a well calibrated model mean?
A model is well calibrated when its predicted probability matches the observed frequency of positive outcomes. For example, if it assigns a probability of 0.8 to a group, about 80 percent of those cases should be positive.
Why are deep neural networks often overconfident?
Studies show that modern neural networks tend to produce confidence scores that are too high. Factors such as depth, width, weight decay, and batch normalization influence the degree of miscalibration.
How can one measure calibration?
Calibration curves plot the observed frequency of positives against predicted probability, using binning to group predictions. The curve shows how close the model's confidence is to reality.
Can calibration be improved after training?
Yes, post-processing methods such as Platt scaling, isotonic regression, and temperature scaling can adjust the probabilities without retraining the model.
Does accuracy capture calibration?
No, accuracy only measures how often the model is correct, not whether its confidence scores align with actual error rates. Calibration is a separate metric evaluated in comprehensive benchmarks.