Trainable Calibration Measures For Neural Networks From Kernel Mean Embeddings

By Aviral Kumar, Sunita Sarawagi, Ujjwal Jain. 2018

344 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Measuring Calibration in Deep Learning (2019, background): “A suite of recent papers (e.g., Lee et al., 2017; Vaicenavicius et al., 2019; Thulasidasan et al., 2019; Kumar et al., 2018; Guo et al., 2017; Seo et al., 2019) use ECE to validate their models’ calibration.”
Angular Visual Hardness (2019, background): “It is well-known that the deep neural networks are mis-calibrated and there has been a rich literature trying to solve this problem [21, 31].”
Calibration tests in multi-class classification: A unifying framework (2019, background): “In fact, as we show in Example I.1, the MMCE is a special case of the KCE.”
Mitigating bias in calibration error estimation (2020, methodology): “, 2017; Zadrozny and Elkan, 2001, 2002) and, more recently, calibration-sensitive training objectives (Karandikar et al., 2021; Krishnan and Tickoo, 2020; Kumar et al., 2018; Lin et al., 2018; Mukhoti et al., 2020).”
Evaluating model calibration in classification (2019, methodology): “(Kumar, Sarawagi, and Jain 2018; Zhang et al. 2018; Kendall and Gal 2017; Tran et al. 2018) proposing methods for obtaining calibrated classifiers.”
Local Temperature Scaling for Probability Calibration (2020, methodology): “2), we show the performance before and after using LTS for models trained with maximum mean calibration loss (MMCE) [36] and focal loss (FL) [52].”
Non-Parametric Calibration for Classification (2019, background): “For example, Pereyra et al. (2017) propose to penalize low-entropy output distributions, Kumar et al. (2018) (Murphy, 1973, DeGroot and Fienberg, 1983) and approaches to provide uncertainty estimates for non-probabilistic binary classiﬁers (Platt, 1999, Lin et al., 2007, Zadrozny and Elkan, 2002).”
Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration (2019, methodology): “An alternative to post-hoc calibration is to modify the classiﬁer learning algorithm itself: MMCE [17] trains neural networks by optimising the combination of log-loss with a kernel-based measure of calibration loss; SWAG [19] models the posterior distribution over the weights of the neural network ”