Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples

By Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin. 2017

963 citations recorded by the source corpus.

The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their predictions, i.e., do not distinguish in- and out-of-distributions. Recently, to handle this issue, several threshold-based detectors have been proposed given pre-trained neural classifiers. However, the performance of prior works highly depends on how to train the classifiers since they only focus on improving inference procedures. In this paper, we develop a novel training method for classifiers so that such inference algorithms can work better. In particular, we suggest two additional terms added to the original loss (e.g., cross entropy). The first one forces samples from out-of-distribution less confident by the classifier and the second one is for (implicitly) generating most effective training samples for the first one. In essence, our method jointly trains both classification and generative neural networks for out-of-distribution. We demonstrate its effectiveness using deep convolutional neural networks on various popular image datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Pretrained Transformers Improve Out-of-Distribution Robustness (2020, background): “Also note that in OOD detection for computer vision, higher accuracy does not reliably improve OOD detection (Lee et al., 2018), so pretrained Transformers’ OOD detection performance is not anticipated.”
Natural Adversarial Examples (2019, background): “Last, we find that classifiers also overgeneralize [21, 22, 33], either over-extrapolating a pattern within a class (shadow→ sundial), or incorrectly including nearby categories (car→ limousine).”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “optimal solution we derived in (11) takes an intuitive form, and has already been used widely as a training objective in many uncertainty works that leverage adversarial training and generative modeling to detect OOD examples (Hafner et al., 2018; Harang and Rudd, 2018; Lee et al., 2018; Malinin and”
Deep Weighted Averaging Classifiers (2018, background): “There have also been several papers focused on the problem of predicting whether data is in-domain or out-of-domain [25, 28].”
Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, methodology): “Lee et al. (2018b) propose Mahalanobis distance-based scores which characterize out-of-distribution samples using hidden features.”
Posterior Meta-Replay for Continual Learning (2021, methodology): “We enforce high uncertainty in OOD inputs by either setting high-entropy softmax labels (i.e., uniform distribution; eg., [38]), or by using different random labels per mini-batch.”
Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild (2019, methodology): “To achieve this, the model should learn to be conﬁdence-calibrated by learning with a certain amount of OOD data as well as data of the previous tasks [19, 20].”
Towards neural networks that provably know when they don't know (2019, methodology): “GAN: The framework of confidence-calibrated classifiers (Lee et al., 2017) relies on training a GAN alongside a classifier such that the GAN’s generator is encouraged to generate points close to but not on the in-distribution.”
Out-of-Domain Detection for Natural Language Understanding in Dialog Systems (2019, methodology): “detection performance is better [14], [15].”
Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem (2018, methodology): “The existing approaches basically either use adjustment techniques of the softmax outputs [8, 21] by temperature rescaling [11] or they use a generative model like a VAE or GAN to model boundary inputs of the true distribution [19, 32] in order to discriminate in-distribution from out-ofdistribution”
Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, methodology): “More recently, Lee et al. (2017) introduce a GAN to generate OOD samples, and Liang et al. (2018) add perturbations to the input, applying an “OOD detector” to improve softmax scores on OOD samples by scaling the temperature.”
Predictive Uncertainty Estimation via Prior Networks (2018, methodology): “One solution is to synthetically generate points on the boundary of the in-domain region using a generative model [21, 22].”
Deep Anomaly Detection with Outlier Exposure (2018, methodology): “Unlike Liang et al. (2018); Lee et al.”
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018, methodology): “…neural classiﬁer trained by standard cross entropy loss typically outperforms the ODIN using softmax neural classiﬁer trained by conﬁdence loss [20] which involves jointly training a generator and a classiﬁer to calibrate the posterior distribution even though training such model is…”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, methodology): “They propose mechanism to calibrate output conﬁdences in deep models (Lee et al., 2018a), formulate a discrepancy loss between knowns and known unknowns (Yu and Aizawa, 2019) or modify the embedding to explicitly separate them.”
Practical Deep Learning with Bayesian Principles (2019, methodology): “We use experimental protocol from the literature [16, 31, 8, 32] to compare VOGN, Adam and MC-dropout on CIFAR-10.”
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty (2019, background): “Lee et al. [2018] expand on this by generating synthetic outliers and training the representations to ﬂag these examples as outliers.”