Mitigating bias in calibration error estimation
By Rebecca Roelofs, Nicholas Cain, Jonathon Shlens, Michael C. Mozer. 2020
131 citations recorded by the source corpus.
Building reliable machine learning systems requires that we correctly understand their level of confidence. Calibration focuses on measuring the degree of accuracy in a model's confidence and most research in calibration focuses on techniques to improve an empirical estimate of calibration error, ECE_bin. Using simulation, we show that ECE_bin can systematically underestimate or overestimate the true calibration error depending on the nature of model miscalibration, the size of the evaluation data set, and the number of bins. Critically, ECE_bin is more strongly biased for perfectly calibrated models. We propose a simple alternative calibration error metric, ECE_sweep, in which the number of bins is chosen to be as large as possible while preserving monotonicity in the calibration function. Evaluating our measure on distributions fit to neural network confidence scores on CIFAR-10, CIFAR-100, and ImageNet, we show that ECE_sweep produces a less biased estimator of calibration error and therefore should be used by any researcher wishing to evaluate the calibration of models trained on similar datasets.
Subfields: Evaluation & Benchmarks, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Evaluation & Benchmarks
- Ekin Dogus Cubuk
- Ludwig Schmidt
- Raphael Gontijo Lopes
- Gabriel Ilharco
- Ali Farhadi
- Yair Carmon
- Mitchell Wortsman
- Rebecca Roelofs
In AI Ethics, Fairness & Society
Citations in the corpus
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, background): “While ensembles improve model calibration [40, 76], model soups do not have the same effect.”
- No One Representation to Rule Them All: Overlapping Features of Training Methods (2021, methodology): “We found it necessary to calibrate all models using temperature scaling (Roelofs et al., 2020; Guo et al., 2017) to maximize ensemble performance.”
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination (2021, background): “As argued by Roelofs et al. (2022) and Dimitriadis et al. (2021), such an assumption is natural, and practitioners tend to dismiss nonisotonic recalibration functions as artifacts.”