Biologically inspired protection of deep networks from adversarial attacks

By Aran Nayebi, Surya Ganguli. 2017

121 citations recorded by the source corpus.

Inspired by biophysical principles underlying nonlinear dendritic computation in neural circuits, we develop a scheme to train deep neural networks to make them robust to adversarial attacks. Our scheme generates highly nonlinear, saturated neural networks that achieve state of the art performance on gradient based adversarial examples on MNIST, despite never being exposed to adversarially chosen examples during training. Moreover, these networks exhibit unprecedented robustness to targeted, iterative schemes for generating adversarial examples, including second-order methods. We further identify principles governing how these networks achieve their robustness, drawing on methods from information geometry. We find these networks progressively create highly flat and compressed internal representations that are sensitive to very few input dimensions, while still solving the task. Moreover, they employ highly kurtotic weight distributions, also found in the brain, and we demonstrate how such kurtosis can protect even linear classifiers from adversarial attack.

Subfields: Artificial Intelligence, Hardware and Architecture, Safety, Robustness & Interpretability, Neuroscience, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

In Safety, Robustness & Interpretability

In Neuroscience

In Signal Processing

Citations in the corpus

Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models (2017, background): “An interesting example is the saturated sigmoid network (Nayebi & Ganguli, 2017) in which an additional regularization term leads the sigmoid activations to saturate, which in turn leads to vanishing gradients and failing gradient-based attacks (Brendel & Bethge, 2017).”
Ensemble Adversarial Training: Attacks and Defenses (2017, background): “For example, Distillation [16] and Saturating Networks [11] introduce numerical instabilities in gradient computations [4, 3] (see Appendix C).”
Stochastic Activation Pruning for Robust Adversarial Defense (2018, background): “Taking a different approach, Nayebi & Ganguli (2017) draws inspiration from biological systems.”
Certified Defenses against Adversarial Examples (2018, background): “…the gradient masking phenomenon (Tramèr et al., 2017; Papernot et al., 2016b) in several proposed defenses that train against approximations like g , such as saturating networks (Nayebi & Ganguli, 2017), distillation (Papernot et al., 2016c), and adversarial training (Goodfellow et al., 2015).”
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples (2017, background): “Some other work has focused on modifying the classiﬁer architecture to increase its robustness, e.g ., Gu & Rigazio (2014), Cisse et al. (2017) and Nayebi & Ganguli (2017).”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “Because many perturbation defense papers we reviewed apply defenses that increase error on the test set [4, 7, 9, 14, 20, 23, 36, 42, 45, 103, 104], these methods are actually more vulnerable to the test set attack than an undefended model.”
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients (2017, methodology): “To simplify comparison, we omit defenses (Xu, Evans, and Qi 2017; Nayebi and Ganguli 2017) that are not fully architecture-agnostic or which work by detecting and rejecting adversarial examples.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Nayebi and Ganguli [124] demonstrated natural robustness of neural networks against adversarial attacks with highly non-linear activations (similar to nonlinear dendritic computations).”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Defenses: Proposed defenses include detection and rejection methods [31, 25, 54, 60, 3, 63], pre-processing, quantization and dimensionality reduction methods [12, 73, 7], manifold-projection methods [39, 72, 82, 86], methods based on stochasticity/regularization or adapted architectures [109, 7, 68”
Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser (2017, background): “Nayebi and Surya [14] use saturating networks for adversarial robustness.”
Comment on "Biologically inspired protection of deep networks from adversarial attacks" (2017, methodology): “A recent paper [1] suggests that Deep Neural Networks can be protected from gradient-based adversarial perturbations by driving the network activations into a highly saturated regime.”
Relating Adversarially Robust Generalization to Flat Minima (2021, methodology): “However, there are also many detection/rejection approaches [34, 29, 55, 58, 2, 60], so-called manifold-projection methods [42, 70, 78, 79], several methods based on pre-processing, quantization and/or dimensionality reduction [7, 71, 5], methods based on randomness, regularization or adapted archit”
Intriguing Properties of Adversarial Examples (2017, background): “Another approach to adversarial examples that considers the linear response of the network can be found in Nayebi & Ganguli (2017).”
Adversarial Examples on Object Recognition (2020, background): “Nayebi and Ganguli [125] developed a new training scheme inspired by biophysical principles in neural circuits.”