Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

By Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel. 2018

524 citations recorded by the source corpus.

The robustness of neural networks to adversarial examples has received great attention due to security implications. Despite various attack approaches to crafting visually imperceptible adversarial examples, little has been developed towards a comprehensive measure of robustness. In this paper, we provide a theoretical justification for converting robustness analysis into a local Lipschitz constant estimation problem, and propose to use the Extreme Value Theory for efficient evaluation. Our analysis yields a novel robustness metric called CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork Robustness. The proposed CLEVER score is attack-agnostic and computationally feasible for large neural networks. Experimental results on various networks, including ResNet, Inception-v3 and MobileNet, show that (i) CLEVER is aligned with the robustness indication measured by the $\ell_2$ and $\ell_\infty$ norms of adversarial examples from powerful attacks, and (ii) defended networks using defensive distillation or bounded ReLU indeed achieve better CLEVER scores. To the best of our knowledge, CLEVER is the first attack-independent robustness metric that can be applied to any neural network classifier.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “Another line of work focuses on finding conservative bounds on the number of robustness violations [48], [57].”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “which prevents the hidden representations h(x) from being overly sensitive to the semantically meaningless perturbations in the pixel space (Ruan et al., 2018; Weng et al., 2018; Tsuzuku et al., 2018; Jacobsen et al., 2018a; Sokolic et al., 2017).”
Certified Adversarial Robustness via Randomized Smoothing (2019, methodology): “…have been proposed in the literature (Wong & Kolter, 2018; Wang et al., 2018a;b; Raghunathan et al., 2018a;b; Wong et al., 2018; Dvijotham et al., 2018b;a; Croce et al., 2018; Gehr et al., 2018; Mirman et al., 2018; Singh et al., 2018; Gowal et al., 2018; Weng et al., 2018a; Zhang et al., 2018).”
Efficient Formal Safety Analysis of Neural Networks (2018, background): “Similarly, existing work on finding lower bounds on the L-norms of adversarial perturbations that can fool a neural network also suffer from the same limitations [22, 31].”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “‘f kx x~k (8) 8 X. Liu, M. Cheng, H. Zhang and C-J. Hsieh for all x;x~. In fact, it has been proved recently that Lipschitz constant can be used to measure the robustness of machine learning model [15,33]. If L ‘f is large enough, even a tiny change of input x x~ can signicantly change the loss an”
CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models (2019, methodology): “, 2018a] [Carlini and Wagner, 2017] [Weng et al., 2018]”
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks (2018, methodology): “We can obtain it in only small networks or get its approximation, which cannot provide certification [12, 31, 9].”
On the Robustness of Interpretability Methods (2018, methodology): “1 This notion has been also used for adversarial attacks on neural networks(Hein & Andriushchenko, 2017; Weng et al., 2018) 2 Naturally, optimizing over l ∞ box constraints is much easier, and thus we take this approach in our experiments.”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, background): “This task has become a common approach to evaluate the robustness of DNNs – generally speaking, the easier an adversarial example can be generated, the less robust the DNN model is.”
A Statistical Approach to Assessing Neural Network Robustness (2018, background): “Weng et al. (2018b) recently constructed a measure for robustness to adversarial attacks estimating a lower bound on the minimum adversarial distortion, that is the smallest perturbation required to create an adversarial example.”
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers (2019, background): “Regularizing Lipschitz continuity stabilizes training, improves the ﬁnal performance, and enhances robustness to adversarial attacks (Weng et al., 2018; Qian & Wegman, 2019).”
MixTrain: Scalable Training of Formally Robust Neural Networks (2018, methodology): “To solve such problem, sound approximations successfully simplify the verification problem with traditional analysis methods like interval [7], convex polytope [26], abstract domain [28], Lagrangian relaxation [29] and Lipschitz continuous [65], [30].”
Do Wider Neural Networks Really Help Adversarial Robustness? (2020, background): “Previous works [29, 64] usually relate local Lipschitzness with network robustness, suggesting that smaller local Lipschitzness leads to robust models.”
Adversarial Robustness Toolbox v0.2.2 (2018, background): “34 9.2 Loss Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 9.3 CLEVER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35”
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks (2019, methodology): “In particular, in [39], the authors convert the robustness analysis problem into a local Lipschitz constant estimation problem, where they estimate this local constant by a set of independently and identically sampled local gradients.”
advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch (2019, background): “…2016; Madry et al., 2017; Cisse et al., 2017; Ding et al., 2018; Wong and Kolter, 2018; Mirman et al., 2018); robustness evaluation methodologies (Weng et al., 2018; Katz et al., 2017; Athalye et al., 2018); and understanding the vulnerability phenomena (Fawzi et al., 2017; Shafahi et al.,…”
Adversarial Robustness Toolbox v1.0.0 (2018, background): “art.metrics.clever u art.metrics.clever t art/metrics/metrics.py The Cross Lipschitz Extreme Value for nEtwork Robustness metric (CLEVER) [52] estimates, for a given input x and (cid:96) p norm, a lower bound γ for the minimal perturbation that is required to change the classiﬁcation of x , i.e.…”