Towards neural networks that provably know when they don't know

By Alexander Meinke, Matthias Hein. 2019

151 citations recorded by the source corpus.

It has recently been shown that ReLU networks produce arbitrarily over-confident predictions far away from the training data. Thus, ReLU networks do not know when they don't know. However, this is a highly important property in safety critical applications. In the context of out-of-distribution detection (OOD) there have been a number of proposals to mitigate this problem but none of them are able to make any mathematical guarantees. In this paper we propose a new approach to OOD which overcomes both problems. Our approach can be used with ReLU networks and provides provably low confidence predictions far away from the training data as well as the first certificates for low confidence predictions in a neighborhood of an out-distribution point. In the experiments we show that state-of-the-art methods fail in this worst-case setting whereas our model can guarantee its performance while retaining state-of-the-art OOD performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Natural Adversarial Examples (2019, background): “, 2019 [41] propose studying adversarial out-of-distribution detection by detecting adversarially optimized uniform noise.”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “optimal solution we derived in (11) takes an intuitive form, and has already been used widely as a training objective in many uncertainty works that leverage adversarial training and generative modeling to detect OOD examples (Hafner et al., 2018; Harang and Rudd, 2018; Lee et al., 2018; Malinin and”
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks (2020, methodology): “We train these networks by following the procedure described by Meinke & Hein (2020) (Appendix C).”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, methodology): “Consequently, many researchers have developed techniques for calibrating the probabilities of classifiers [473], [487]–[492] or Bayesian uncertainty quantification [417], [418], [472], [474], [475], [493].”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Enhancement [69], [112], [113], [114], [115], [116], [117], [118], [119], [120], [121], [122], [123], [124], [125]”
CODEs: Chamfer Out-of-Distribution Examples against Overconfidence Issue (2021, methodology): “We train LeNet on MNIST and FMNIST while ResNet-18 for CIFAR-10, CIFAR-100 and SVHN, and then evaluate them on the corresponding test set to report the test error (TE), and on both in-and out-of-distribution datasets to report mean maximal confidence (MMC) following [34].”
Out-of-distribution Detection with Deep Nearest Neighbors (2022, unknown): “…(Lee et al., 2017; Bevandi ´ c et al., 2018; Malinin & Gales, 2018; Hendrycks et al., 2019; Geifman & El-Yaniv, 2019; Hein et al., 2019; Meinke & Hein, 2019; Mohseni et al., 2020; Liu et al., 2020; Jeong & Kim, 2020; Van Amersfoort et al., 2020; Yang et al., 2021; Chen et al., 2021;…”