Understanding Measures of Uncertainty for Adversarial Example Detection

By Lewis Smith, Yarin Gal. 2018

421 citations recorded by the source corpus.

Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence. But many measures of uncertainty exist, including predictive en- tropy and mutual information, each capturing different types of uncertainty. We study these measures, and shed light on why mutual information seems to be effective at the task of adversarial example detection. We highlight failure modes for MC dropout, a widely used approach for estimating uncertainty in deep models. This leads to an improved understanding of the drawbacks of current methods, and a proposal to improve the quality of uncertainty estimates using probabilistic model ensembles. We give illustrative experiments using MNIST to demonstrate the intuition underlying the different measures of uncertainty, as well as experiments on a real world Kaggle dogs vs cats classification dataset.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Posterior Meta-Replay for Continual Learning (2021, methodology): “For C -way classiﬁcation, we compute this quantity as the average standard deviation of predicted likelihood values for K models w ( t,k ) ∼ q θ ( t ) ( W ) [67]: where SD ( · ) refers to the standard deviation of the given set of values.”
Prior Networks for Detection of Adversarial Attacks (2018, methodology): “Detection of adversarial attack using Monte-Carlo dropout was further investigated in [21].”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “[313], Malinin and Gales [314], Jain et al.”
Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection (2020, methodology): “Our analysis involved a comparison of two variational-dropweights based uncertainty measures, Predictive Entropy (PH) and Bayesian Active Learning by Disagreement (BALD) [13, 17] , in their application to COVID-19 image classification.”
U-CAM: Visual Explanation Using Uncertainty Based Class Activation Maps (2019, background): “[23, 24, 43] have mainly decomposed predictive uncertainty into two major types, namely aleatoric and epistemic uncertainty, which capture uncertainty about the predicted model and uncertainty present in the data itself.”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, positive): “Typically, data augmentation can also be employed during test time to boost prediction accuracy [109], [110], estimate uncertainty [111], and enhance robustness [112], [113].”
Selective Question Answering under Domain Shift (2020, background): “Second, we take the negative variance of the ˆ p i (ˆ y ) ’s (Feinman et al., 2017; Smith and Gal, 2018): Higher variance corresponds to greater uncertainty, and hence favors abstaining.”
Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, background): “Aleatoric uncertainty is uncertainty inherent in the data such as an image of a 3 that is similar to an 8 (Smith & Gal, 2018).”
Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty (2021, background): “As mentioned in Gal [2016], Smith and Gal [2018], the predictive entropy H [ Y | x , D ] of p ( y | x , D ) upper-bounds the epistemic uncertainty, given as the mutual information I [ Y ; ω | x , D ] ( information gain ) between parameters ω and output y , following the equation: .”
VectorDefense: Vectorization as a Defense to Adversarial Examples (2018, background): “Recent research suggests that the adversarial problem is an artifact of classiﬁers being imperfectly trained i.e. having a non-zero error on test data [9, 35].”
Ensemble Distribution Distillation (2019, background): “Ensembles of NNs are known to yield increased accuracy over single models [13], allow useful measures of uncertainty to be derived [11] and also provide defense against adversarial attack [14].”
Are Generative Classifiers More Robust to Adversarial Attacks? (2018, background): “In addition, recent advances in Bayesian neural networks have demonstrated that uncertainty estimates can be used to detect adversarial attacks (Li & Gal, 2017; Feinman et al., 2017; Louizos & Welling, 2017; Smith & Gal, 2018).”
A survey of uncertainty in deep neural networks (2021, background): “it is often badly calibrated [19] and does not have any information about the certainty the model itself has on this specific output [19].”