Ground-Truth Adversarial Examples

By Nicholas Carlini, Guy Katz, Clark W. Barrett, David L. Dill. 2017

91 citations recorded by the source corpus.

The ability to deploy neural networks in real-world, safety-critical systems is severely limited by the presence of adversarial examples: slightly perturbed inputs that are misclassified by the network. In recent years, several techniques have been proposed for training networks that are robust to such examples; and each time stronger attacks have been devised, demonstrating the shortcomings of existing defenses. This highlights a key difficulty in designing an effective defense: the inability to assess a network's robustness against future attacks. We propose to address this difficulty through formal verification techniques. We construct ground truths: adversarial examples with a provably-minimal distance from a given input point. We demonstrate how ground truths can serve to assess the effectiveness of attack techniques, by comparing the adversarial examples produced by those attacks to the ground truths; and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement. We use this technique to assess recently suggested attack and defense techniques.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “Algorithms have been developed that search for the smallest additive distortions in RGB space that are sufﬁcient to confuse a classiﬁer (Carlini et al., 2017).”
Towards Deep Learning Models Resistant to Adversarial Attacks (2017, background): “Furthermore, recent work (Carlini et al., 2017) on verifiable adversarial examples showed that our proposed defense reliably increased the robustness to any `∞-bounded attack.”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “Only recently, it has been demonstrated [19, 3, 22] that robust optimization approaches (also known as adversarial training) can reliably increase the resilience of neural network classifiers to a broad class of adversarial examples.”
Formal Security Analysis of Neural Networks using Symbolic Intervals (2018, background): “To tackle these challenges, all prior work on the formal security analysis of neural networks [6, 9, 16, 20] rely on different types of Satisfiability Modulo Theories (SMT) solvers and are thus severely limited by the efficiency of the solvers.”
Certified Defenses against Adversarial Examples (2018, methodology): “Another set of numbers comes from Carlini et al. (2017), who use formal veriﬁcation methods to compute A opt exactly on 10 input examples for a small ( 72 -node) variant of the Madry et al. network.”
Verifying Neural Networks with Mixed Integer Programming (2017, positive): “Carlini et al . introduce the idea of using ground-truth adversarial examples [3] to design future-proof defenses.”
Provable defenses against adversarial examples via the convex outer adversarial polytope (2017, methodology): “The ﬁrst (which is a topic that is broader than merely adversarial examples), is the work in the veriﬁcation of neural networks [Huang et al., 2017, Katz et al., 2017, Ehlers, 2017].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “`p(p ≥ 0) `0: [72], [75], `1: [79], `2: [19], [70]–[74], [76]–[78], [80], [82], `∞: [71], [72], [74], [79], [82], PASS [68] None [83]”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Recently, Carlini and Wagner [59] also demonstrated that ten different defenses against perturbations can again be defeated by new attacks constructed using new loss functions.”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, methodology): “In this vein, we note the growing community on veriﬁcation of neural networks (Katz et al., 2017; Bunel et al., 2017; Carlini et al., 2017), which provide techniques to compute provably worst-case adversarial examples.”
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models (2018, methodology): “made: from complete methods that use Satisfiability Modulo Theory (SMT) [4, 8, 11] or MixedInteger Programming (MIP) [1, 5, 17] to incomplete methods that rely on solving a convex relaxation of the verification problem [6, 7, 9, 15, 18, 19, 20].”
Semidefinite relaxations for certifying robustness to adversarial examples (2018, methodology): “The second category draws techniques from formal verification such as SMT [16, 17, 7, 14], which aim to provide tight certificates for any network using discrete optimization.”
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation (2019, methodology): “, 2017) or Satisfiability Modulo Theory (SMT) (Katz et al., 2017; Carlini et al., 2017), and incomplete methods that solve a convex relaxation of the verification problem (Weng et al.”
An abstract domain for certifying neural networks (2019, result): “While this was effective in experiments, Carlini et al. [2017] demonstrated an attack for the safety-critical problem of ground-truthing, where this defense occasionally exacerbated the problem.”
Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models (2018, background): “There are other types of adversarial examples like [30], and other generating approaches including GAN attacks [40], ensemble attacks [26], ground truth attacks [6], hot/cold attacks [35], feature adversary [36] which we do not present further details.”
Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability (2018, methodology): “Exact verification of networks has been the subject of many recent works [15, 9, 3, 30, 19, 6].”
Explainable Deep Learning: A Field Guide for the Uninitiated (2020, background): “, 2019) and white-box (Szegedy et al., 2013; Goodfellow et al., 2014; Sabour et al., 2015; Nguyen et al., 2015; Kurakin et al., 2016; Rozsa et al., 2016; Papernot et al., 2016a; Moosavi-Dezfooli et al., 2016; Tabacof & Valle, 2016; Kurakin et al., 2016; Carlini & Wagner, 2017; Moosavi-Dezfooli et al”
Global Robustness Evaluation of Deep Neural Networks with Provable Guarantees for L0 Norm (2018, methodology): “Then, we apply the mode-1 tensor unfolding operation to have T[1](T (T0, t)) ∈ Rn×M such that M = ∆ · p · k.”
DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks (2017, methodology): “Reluplex has been used in evaluating techniques for finding and defending against adversarial perturbations [2], and it has also been successfully applied to a real-world family of deep neural networks, designed to operate as controllers in the next-generation Airborne Collision Avoidance System for”