Adversarial Vulnerability of Neural Networks Increases With Input Dimension

By Carl-Johann Simon-Gabriel, Yann Ollivier, Bernhard Schölkopf, L. Eon Bottou, David Lopez-Paz. 2018

49 citations recorded by the source corpus.

Over the past four years, neural networks have proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when seen as a function of the inputs. For most current network architectures, we prove that the $\ell_1$-norm of these gradients grows as the square root of the input-size. These nets therefore become increasingly vulnerable with growing image size. Over the course of our analysis we rediscover and generalize double-backpropagation, a technique that penalizes large gradients in the loss surface to reduce adversarial vulnerability and increase generalization performance. We show that this regularization-scheme is equivalent at first order to training with adversarial noise. Finally, we demonstrate that replacing strided by average-pooling layers decreases adversarial vulnerability. Our proofs rely on the network's weight-distribution at initialization, but extensive experiments confirm their conclusions after training.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, background): “Further connections have been drawn between adversarial training and regularizing the gradients of the network’s activations around the data (Simon-Gabriel et al., 2018).”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “Since the seminal work of [42] exhibiting this intriguing phenomenon in the context of deep learning, a wealth of results have been published on designing attacks [18, 34, 32, 23, 6, 31] and defenses [18, 35, 20, 29, 39, 27]), or on trying to understand the very nature of this phenomenon [17, 40, 15”
Sorting out Lipschitz function approximation (2018, background): “In fact, adversarial training can be viewed as approximate gradient regularization [32, 42] which makes the function Lipschitz locally around the training data.”
Are adversarial examples inevitable? (2018, background): “Simon-Gabriel et al. (2018) show that the Lipschitz constant of untrained networks with random weights gets large in high dimensions.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Generalization: Research also includes independent benchmarks of attacks and defenses [13, 15, 5, 6, 85], their properties [57, 84], as well as theoretical questions [34, 42, 23, 99, 27, 88, 102, 104].”
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks (2018, background): “, via regularization, may allow us to learn more robust classifiers not only against evasion [22, 35, 39, 44] but also against poisoning availability attacks.”
On the Connection Between Adversarial Robustness and Saliency Map Interpretability (2019, background): “As (Simon-Gabriel et al., 2018) demonstrate, double backpropagation makes neural networks more resilient to adversarial attacks.”
Batch Normalization is a Cause of Adversarial Vulnerability (2019, methodology): “The frequently made observation that adversarial vulnerability can scale with the input dimension (Goodfellow et al., 2015; Gilmer et al., 2018; Simon-Gabriel et al., 2018) highlights the importance of identifying regularizers as more than merely a way to improve test accuracy.”
A Kernel Perspective for Regularizing Deep Neural Networks (2018, methodology): “, 2018), gradient penalties and double backpropagation (Drucker and Le Cun, 1991; Simon-Gabriel et al., 2018; Gulrajani et al., 2017; Roth et al., 2017, 2018; Arbel et al., 2018), adversarial training (Madry et al.”
Relating Adversarially Robust Generalization to Flat Minima (2021, methodology): “2 [112] Carl-Johann Simon-Gabriel, Yann Ollivier, Bernhard Schölkopf, Léon Bottou, and David Lopez-Paz.”
On Regularization and Robustness of Deep Neural Networks (2018, methodology): “Related penalties have been considered in the context of robust optimization, for regularization or robustness, noting that a penalty based on the gradient of the loss can give a good approximation of (7) when is small (Drucker & Le Cun, 1991; Lyu et al., 2015; Roth et al., 2018; Simon-Gabriel et al”