Stochastic Activation Pruning for Robust Adversarial Defense

By Guneet Singh Dhillon, Kamyar Azizzadenesheli, Zachary Chase Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, Anima Anandkumar. 2018

581 citations recorded by the source corpus.

Neural networks are known to be vulnerable to adversarial examples. Carefully chosen perturbations to real images, while imperceptible to humans, induce misclassification and threaten the reliability of deep learning systems in the wild. To guard against adversarial examples, we take inspiration from game theory and cast the problem as a minimax zero-sum game between the adversary and the model. In general, for such games, the optimal strategy for both players requires a stochastic policy, also known as a mixed strategy. In this light, we propose Stochastic Activation Pruning (SAP), a mixed strategy for adversarial defense. SAP prunes a random subset of activations (preferentially pruning those with smaller magnitude) and scales up the survivors to compensate. We can apply SAP to pretrained networks, including adversarially trained models, without fine-tuning, providing robustness against adversarial examples. Experiments demonstrate that SAP confers robustness against attacks, increasing accuracy and preserving calibration.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

GAN Inversion: A Survey (2021, background): “In contrast, adversarial defense [152], [153] aims at preventing the model from being fooled by attackers.”
Adversarial Example Games (2020, methodology): “On the other hand obfuscation methods rely on masking the input gradient needed by an attacker to construct adversarial examples [60, 10, 32, 18].”
Adversarial Training for Free! (2019, background): “…et al., 2018], PixelDefend [Song et al., 2017], Robust Manifold Defense [Ilyas et al., 2017], L2-nonexpansive nets [Qian and Wegman, 2018], Jacobian Regularization [Jakubovitz and Giryes, 2018], Universal Perturbation [Shafahi et al., 2018], and Stochastic Activation Pruning [Dhillon et al., 2018].”
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks (2018, background): “A variety of defenses against adversarial inputs have been proposed [37,13] and broken [20,9,5]; research into defenses that provide strong guarantees of robustness is ongoing.”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, methodology): “Noise can be drawn either from Gaussian [26, 24, 37], Laplace [24], Uniform [44], or Multinomial [12] distributions.”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “ model that is robust to the rst order attacks, but this approach does not work very well on more dicult datasets such as CIFAR-10. It is worth mentioning that there are many defense algorithms (r.f.[3,19,13,8,36,27,25]) against white box attacks in literature. Unfortunately, as [2,1] pointed out, t”
Towards the first adversarially robust neural network model on MNIST (2018, background): “• Stochasticity: Certain inputs or hidden activations are shuffled or randomized (Prakash et al., 2018; Dhillon et al., 2018; Xie et al., 2018).”
Are adversarial examples inevitable? (2018, background): “…dimensionality (Ma et al., 2018), input transformations such as compression and image quilting (Guo et al., 2017), stochastic activation pruning (Dhillon et al., 2018), adding randomization at inference time (Xie et al., 2017), enhancing the conﬁdence of image labels (Song et al., 2017), and…”
Efficient Exploration Through Bayesian Deep Q-Networks (2018, background): “It has been observed that drop-out results in the ensemble of inﬁnitely many models but all models almost the same (Dhillon et al., 2018; Osband et al., 2016).”
Adversarial Robustness through Local Linearization (2019, background): “Of the many approaches proposed [19, 2, 6, 21, 15, 17], adversarial training [14, 16] is empirically the best performing algorithm to train networks robust to adversarial perturbations.”
Improving Transferability of Adversarial Examples With Input Diversity (2018, background): “[7] pruned a random subset of activations according to their magnitude to enhance network robustness.”
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (2020, background): “Approaches range from adding stochasticity [7], to label smoothening and feature squeezing [27, 38], to denoising and training on adversarial examples [22, 19].”
Adversarial Robustness with Semi-Infinite Constrained Learning (2021, background): “[45] Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar.”
Generative Adversarial Perturbations (2017, background): “While our focus is on efficient attacks, we refer the reader to [33, 57, 19, 47, 32, 50, 48, 54, 3, 13, 44, 16, 55, 45, 42, 56] for recent works on defense.”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “Some examples of this strategy include the work in (Dhillon et al., 2018) that randomly drops some neurons of each layer of the DNN model, and the work in (Xie et al., 2017a) that resizes the input images to a random size and pads zeros around the input image.”