Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors

By Alireza Shafaei, Mark W. Schmidt, James J. Little. 2018

61 citations recorded by the source corpus.

In the real world, a learning system could receive an input that looks nothing like anything it has seen during training, and this can lead to unpredictable behaviour. We thus need to know whether any given input belongs to the population distribution of the training data to prevent unpredictable behaviour in deployed systems. A recent surge of interest on this problem has led to the development of sophisticated techniques in the deep learning literature. However, due to the absence of a standardized problem formulation or an exhaustive evaluation, it is not evident if we can rely on these methods in practice. What makes this problem different from a typical supervised learning setting is that we cannot model the diversity of out-of-distribution samples in practice. The distribution of outliers used in training may not be the same as the distribution of outliers encountered in the application. Therefore, classical approaches that learn inliers vs. outliers with only two datasets can yield optimistic results. We introduce OD-test, a three-dataset evaluation scheme as a practical and more reliable strategy to assess progress on this problem. The OD-test benchmark provides a straightforward means of comparison for methods that address the out-of-distribution sample detection problem. We present an exhaustive evaluation of a broad set of methods from related areas on image classification tasks. Furthermore, we show that for realistic applications of high-dimensional images, the existing methods have low accuracy. Our analysis reveals areas of strength and weakness of each method.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

Scalable agent alignment via reward modeling: a research direction (2018, background): “Well-calibrated models could engage risk-averse decision making, but handling out-of-distribution states reliably would require higher quality uncertainty estimates than current deep learning techniques can provide (Shafaei et al., 2018).”
Towards neural networks that provably know when they don't know (2019, methodology): “We take the softmax from 7 forward passes (Shafaei et al., 2018) and use the mean of the output for prediction and the variance as score.”
Out-of-Domain Detection for Natural Language Understanding in Dialog Systems (2019, methodology): “Various methods have been proposed to improve the OOD detection performance of neural NLU models [6]–[10], and most of them follow a threshold-based protocol.”
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift (2019, methodology): “We refer to Shafaei et al. (2018) for a recent summary of these methods.”
Do Deep Generative Models Know What They Don't Know? (2018, background): “Shafaei et al. (2018) observe that PixelCNN++ cannot provide reliable outlier detection.”
Likelihood Ratios for Out-of-Distribution Detection (2019, methodology): “…methods for OOD detection and show that the likelihood of auto-regressive models can be confounded by background statistics, providing an explanation to the failure of PixelCNN for OOD detection observed by recent work (Nalisnick et al., 2018; Hendrycks et al., 2018; Shafaei et al., 2018).”
Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality (2019, background): “Recent work [37, 9, 52] showed that a variety of deep generative models fail to distinguish training from out-of-distribution (OOD) data according to the model likelihood.”
COVID-19 Image Data Collection: Prospective Predictions Are the Future (2020, methodology): “Out-of-distribution (OoD) tools [Shafaei et al., 2018] or unpaired distribution matching models [Zhu & Park, 2017; Kim et al.”
Detecting Out-of-Distribution Inputs to Deep Generative Models Using a Test for Typicality (2019, background): “Recent work [26, 8, 32, 19, 34] has shown that a variety of types of deep generative models fail to distinguish training data from out-of-distribution (OOD) inputs according to the model likelihood.”
Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection (2019, unknown): “…training procedure, or computed/extracted post-hoc from the model and/or predictions (DeVries & Taylor, 2018; An & Cho, 2015; S¨olch et al., 2016; Ahmed & Courville, 2019; Sricharan & Srivastava, 2018; Hendrycks & Gimpel, 2016; Hendrycks et al., 2018; Liang et al., 2017; Shafaei et al., 2018).”