Certified Defenses against Adversarial Examples

By Aditi Raghunathan, Jacob Steinhardt, Percy Liang. 2018

1,019 citations recorded by the source corpus.

While neural networks have achieved high accuracy on standard image classification benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbations to test inputs. Defenses based on regularization and adversarial training have been proposed, but often followed by new, stronger attacks that defeat these defenses. Can we somehow end this arms race? In this work, we study this problem for neural networks with one hidden layer. We first propose a method based on a semidefinite relaxation that outputs a certificate that for a given network and test input, no attack can force the error to exceed a certain value. Second, as this certificate is differentiable, we jointly optimize it with the network parameters, providing an adaptive regularizer that encourages robustness against all attacks. On MNIST, our approach produces a network and a certificate that no attack that perturbs each pixel by at most \epsilon = 0.1 can cause more than 35% test error.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, neutral): “, see [13, 28, 34, 55, 57, 66, 93, 96, 105, 114, 115] among many others.”
Adversarial Examples Are Not Bugs, They Are Features (2019, background): “While there might exist such fundamental barriers to robustly classifying standard datasets, this model cannot fully explain the situation observed in practice, where one can train (reasonably) robust classiﬁers on standard datasets [Mad+18; RSL18; WK18; Xia+19; CRK19].”
Unsolved Problems in ML Safety (2021, background): “Researchers could improve adversarial robustness certifications [144, 116, 37], so that models have verifiable adversarial robustness.”
Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “network training process to minimize the number of robustness violations [30], [50], [12].”
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “Increasing MNIST models’ robustness to such perturbations has since become a standard benchmark (Schott et al., 2019; Madry et al., 2017; Wong & Kolter, 2018; Raghunathan et al., 2018).”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, methodology): “We consider the l ∞ -ball adversarial constraint throughout the paper, as it is widely adopted by most adversarial defense methods [16, 33, 34, 40, 50, 61, 66].”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “Potential (costly) approaches include training on gradient-free attacks, or extending certified defenses [28, 42] to multiple perturbations.”
Adversarial Training for Free! (2019, background): “Recently, there has been a surge of certiﬁed defenses [Wong and Kolter, 2017, Wong et al., 2018, Raghunathan et al., 2018a,b, Wang et al., 2018].”
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness (2019, methodology): “However, several years later, it is now argued that training MNIST classifiers whose decision is constant in an `p-norm ball around their training data provides robustness to adversarial examples (Schott et al., 2019; Madry et al., 2017; Wong & Kolter, 2018; Raghunathan et al., 2018).”
Data Poisoning against Differentially-Private Learners: Attacks and Defenses (2019, positive): “We are not the first to design defenses to data poisoning [Steinhardt et al., 2017; Raghunathan et al., 2018], but our proposal has several notable strengths compared to previous work.”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “Designing models that are robust to such attacks, as well as coming up with more effective attacks, is an active area of research (Papernot et al., 2016c; Madry et al., 2017; Tramèr et al., 2017; Wong and Kolter, 2018; Raghunathan et al., 2018; Sinha et al., 2018; Athalye et al., 2018; Papernot and ”
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples (2018, methodology): “We omit two defenses with provable security claims (Raghunathan et al., 2018; Sinha et al., 2018) and one that only argues black-box security (Tramèr et al.”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “, 2009) and has recently been applied successfully in the context of defending neural networks against adversarial examples (Madry et al., 2017; Sinha et al., 2017; Raghunathan et al., 2018; Kolter & Wong, 2017).”
Ensemble Adversarial Training: Attacks and Defenses (2017, background): “More-over, recent works by Sinha et al. (2018), Raghunathan et al. (2018) and Kolter & Wong (2017) even succeed in providing certiﬁable robustness for small perturbations on MNIST.”
Certified Adversarial Robustness via Randomized Smoothing (2019, methodology): “The latter approach has been found more effective (Wong & Kolter, 2018; Raghunathan et al., 2018a).”
Efficient Formal Safety Analysis of Neural Networks (2018, methodology): “While these techniques tend to scale significantly better than solver-based approaches, they suffer from high false positive rates and struggle to find concrete counterexamples demonstrating violations of safety properties [32, 26, 9].”
Formal Security Analysis of Neural Networks using Symbolic Intervals (2018, methodology): “[44] transform the verification problem into a convex optimization problem using relaxations to over-approximate the outputs of ReLU nodes.”