Verified Uncertainty Calibration

By Ananya Kumar, Percy Liang, Tengyu Ma. 2019

468 citations recorded by the source corpus.

Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a prediction. Most models are not calibrated out of the box but are recalibrated by post-processing model outputs. We find in this work that popular recalibration methods like Platt scaling and temperature scaling are (i) less calibrated than reported, and (ii) current techniques cannot estimate how miscalibrated they are. An alternative method, histogram binning, has measurable calibration error but is sample inefficient---it requires $O(B/\epsilon^2)$ samples, compared to $O(1/\epsilon^2)$ for scaling methods, where $B$ is the number of distinct probabilities the model can output. To get the best of both worlds, we introduce the scaling-binning calibrator, which first fits a parametric function to reduce variance and then bins the function values to actually ensure calibration. This requires only $O(1/\epsilon^2 + B)$ samples. Next, we show that we can estimate a model's calibration error more accurately using an estimator from the meteorological community---or equivalently measure its calibration error with fewer samples ($O(\sqrt{B})$ instead of $O(B)$). We validate our approach with multiclass calibration experiments on CIFAR-10 and ImageNet, where we obtain a 35% lower calibration error than histogram binning and, unlike scaling methods, guarantees on true calibration. In these experiments, we also estimate the calibration error and ECE more accurately than the commonly used plugin estimators. We implement all these methods in a Python library: this https URL

Subfields: Artificial Intelligence, Statistics and Probability, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Massive Multitask Language Understanding (2020, background): “Another calibration measure is the Root Mean Squared (RMS) calibration error (Hendrycks et al., 2019a; Kumar et al., 2019).”
Natural Adversarial Examples (2019, methodology): “We estimate the RMS Calibration Error using adaptive binning [36, 22, 17].”
Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement (2023, methodology): “To understand why ImageNet models are significantly more robust than ImageNet models we evaluate their Expected Calibration Error (ECE) [32] on the validation set.”
Unsolved Problems in ML Safety (2021, background): “To help models express their domain of competence in a more representative and meaningful way, researchers could further improve model calibration on typical testing data [68, 131, 112, 108, 205, 106, 107, 123], though the greater challenge is calibration on testing data that is unlike the training ”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “, the maximum probability score) and its actual accuracy [29, 40].”
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift (2022, background): “Calibration has been widely studied in machine learning [Naeini et al., 2014, Guo et al., 2017, Kumar et al., 2019], and applications such as meteorology [Murphy, 1973, DeGroot and Fienberg, 1983, Gneiting and Raftery, 2005], fairness [Hebert-Johnson et al., 2018], and health-care [Jiang et al.,…”
Revisiting the Calibration of Modern Neural Networks (2021, background): “The first of these biases is always negative (Kumar et al., 2019), while the second one is always positive.”
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning (2020, background): “A recent line of works on measuring calibration in deep learning (Vaicenavicius et al., 2019; Kumar et al., 2019; Nixon et al., 2019) outlines several problems of the ECE score.”
Understanding the Effects of Iterative Prompting on Truthfulness (2024, methodology): “For the calibration analysis, we employ ten bins and utilize the standard definition of Expected Calibration Error (ECE), as defined in [40], to calculate the calibration error.”
Classification with Valid and Adaptive Coverage (2020, background): “A different but related line of work focuses on post-processing the output of black-box classification algorithms to produce more accurate probability estimates [6, 11, 15, 16, 22, 27, 28], although without achieving prediction sets with provable finite-sample coverage.”
Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination (2021, background): “Recently, the estimation of calibration error has seen a surge of interest in machine learning (Guo et al., 2017; Kuleshov et al., 2018; Kumar et al., 2019; Nixon et al., 2019; Roelofs et al., 2022).”
Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration (2019, methodology): “A similar measure of classwise-calibration called L2 marginal calibration error was proposed in a concurrent work by [16].”
PAC Confidence Predictions for Deep Neural Network Classifiers (2020, background): “One approach is to provide guarantees in expectation over x (Kumar et al., 2019); however, this approach does not provide guarantees for individual predictions.”
Calibrating Deep Neural Networks using Focal Loss (2020, methodology): “As shown in Figure 4, we also compute 90% confidence intervals for ECE, AdaECE and Classwise-ECE using 1000 bootstrap samples following [15], and using ResNet-50/110 trained on CIFAR-10 (see Appendix G for more results).”
MoPro: Webly Supervised Learning with Momentum Prototypes (2020, methodology): “We report both accuracy and the `2 calibration error (Kumar et al., 2019).”
Mitigating bias in calibration error estimation (2020, methodology): “Recent work observed that the calculation of ECEbin is sensitive to implementation (Kumar et al., 2019; Nixon et al., 2019).”
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving (2020, methodology): “[82] proposed the Marginal Calibration Error (MCE) metric, as an extension of ACE.”
Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models (2021, unknown): “, 2017), and scaling-binning calibrator (Scal-bin) (Kumar et al., 2019).”