Energy-Based Open-World Uncertainty Modeling for Confidence Calibration
By Yezhen Wang, Bo Li, Tong Che, Kaiyang Zhou, Ziwei Liu, Dongsheng Li. 2021
80 citations recorded by the source corpus.
Confidence calibration is of great importance to the reliability of decisions made by machine learning systems. However, discriminative classifiers based on deep neural networks are often criticized for producing overconfident predictions that fail to reflect the true correctness likelihood of classification accuracy. We argue that such an inability to model uncertainty is mainly caused by the closed-world nature in softmax: a model trained by the cross-entropy loss will be forced to classify input into one of K pre-defined categories with high probability. To address this problem, we for the first time propose a novel K+1-way softmax formulation, which incorporates the modeling of open-world uncertainty as the extra dimension. To unify the learning of the original K-way classification task and the extra dimension that models uncertainty, we 1) propose a novel energy-based objective function, and moreover, 2) theoretically prove that optimizing such an objective essentially forces the extra dimension to capture the marginal data distribution. Extensive experiments show that our approach, Energy-based Open-World Softmax (EOW-Softmax), is superior to existing state-of-the-art methods in improving confidence calibration.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ziwei Liu
- Fahad Shahbaz Khan
- Yixuan Li
- Kaiyang Zhou
- Salman Hameed Khan
- Jingkang Yang
- K. J. Joseph
- Vineeth Nallure Balasubramanian
Citations in the corpus
- Energy-based Latent Aligner for Incremental Learning (2022, background): “[49] propose an energy-based open-world softmax objective that can jointly perform discriminative learning and generative modeling.”
- Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Moreover, OOD detection encompasses a broader spectrum of learning tasks (e.g.,multi-label classification (Wang et al., 2021), object detection (Du et al., 2022b,a)) and solution space.”
- OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, methodology): “methods take the major part of OOD detectors, which gets confidence directly from the classifier with some design that is beneficial to OOD detection [9, 10].”