A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
By Dan Hendrycks, Kevin Gimpel. 2016
4,454 citations recorded by the source corpus.
We consider the two related problems of detecting if an example is misclassified or out-of-distribution. We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection. We assess performance by defining several tasks in computer vision, natural language processing, and automatic speech recognition, showing the effectiveness of this baseline across all. We then show the baseline can sometimes be surpassed, demonstrating the room for future research on these underexplored detection tasks.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Sergey Levine
- Percy Liang
- Quoc V. Le
- Jianfeng Gao
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
In Safety, Robustness & Interpretability
Citations in the corpus
- Pretrained Transformers Improve Out-of-Distribution Robustness (2020, methodology): “the-art OOD detection technique is to use the model’s prediction confidence to separate in- and out-of-distribution examples (Hendrycks and Gimpel, 2017).”
- SpanBERT: Improving Pre-training by Representing and Predicting Spans (2019, methodology): “…token pi: yi = f(xs−1,xe+1,pi) In this work, we implement the representation function f(·) as a 2-layer feed-forward network with GeLU activations (Hendrycks and Gimpel, 2016) and layer normalization (Ba et al., 2016): h = LayerNorm(GeLU(W1 · [xs−1;xe+1;pi])) f(·) = LayerNorm(GeLU(W2 ·…”
- Measuring Mathematical Problem Solving With the MATH Dataset (2021, methodology): “Following Hendrycks and Gimpel (2017), we computed the probability that a correct answer has higher confidence than an incorrect answer.”
- Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “…has focused on the important challenges of robustness to adversarial examples (Szegedy et al., 2014; Carlini & Wagner, 2017; 2016), unknown unknowns (Hendrycks et al., 2019; Hendrycks & Gimpel, 2017b; Liu et al., 2018), and model or data poisoning (Steinhardt et al., 2017; Hendrycks et al., 2018).”
- Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, background): “However, a self-driving car needs to correctly estimate at least some attributes of an incoming, unknown object (instead of simply rejecting it [17, 38]) to handle the situation gracefully and minimize damage.”
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “CutMix also enhances the model robustness and alleviates the over-confidence issue [13, 21] of deep networks.”
- Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, methodology): “Typically for evaluation, threshold-free metrics are used, such as the area under the receiver-operating curve (AUROC) (Hendrycks & Gimpel, 2016).”
- Scalable agent alignment via reward modeling: a research direction (2018, background): “The agent can defer to the human or fall back to risk-averse decision making when uncertainty is large, for instance on inputs that do not resemble the training distribution (Hendrycks & Gimpel, 2017).”
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, methodology): “Spurred by this challenge, we propose an anomaly detection task (Hendrycks & Gimpel, 2017; Tack et al., 2020): Can we detect when the true reward starts to drop, while maintaining a low false positive rate in benign cases?”
- Aligning AI With Shared Human Values (2020, methodology): “To measure this, we follow Hendrycks and Gimpel (2017) and use the Area Under the Receiver Operating Characteristic curve (AUROC), where 50% is random chance performance.”
- Unsolved Problems in ML Safety (2021, background): “Anomaly detection is actively studied in research areas such as out-of-distribution detection [79], open-set detection [10], and one-class learning [178, 82], but many challenges remain.”
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (2016, background): “Concurrent with our work, Hendrycks and Gimpel [23] and Guo et al.”
- Early Methods for Detecting Adversarial Images (2016, methodology): “Finally, any general work on out-of-distribution detection or error detection (Hendrycks & Gimpel, 2017) may transfer directly to this adversarial setting.”