Comment on "Biologically inspired protection of deep networks from adversarial attacks"

By Wieland Brendel, Matthias Bethge. 2017

34 citations recorded by the source corpus.

A recent paper suggests that Deep Neural Networks can be protected from gradient-based adversarial perturbations by driving the network activations into a highly saturated regime. Here we analyse such saturated networks and show that the attacks fail due to numerical limitations in the gradient computations. A simple stabilisation of the gradient estimates enables successful and efficient attacks. Thus, it has yet to be shown that the robustness observed in highly saturated networks is not simply due to numerical limitations.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models (2017, background): “An interesting example is the saturated sigmoid network (Nayebi & Ganguli, 2017) in which an additional regularization term leads the sigmoid activations to saturate, which in turn leads to vanishing gradients and failing gradient-based attacks (Brendel & Bethge, 2017).”
Ensemble Adversarial Training: Attacks and Defenses (2017, background): “Given the ubiquity of this pitfall among proposed defenses against adversarial examples [4, 3, 15], we advise researchers to always consider both white-box and black-box adversaries when evaluating defensive strategies.”
Towards the first adversarially robust neural network model on MNIST (2018, background): “There has been much work showing that basically all defenses suggested so far in the literature do not substantially increase robustness over undefended neural networks (Athalye et al., 2018; Brendel & Bethge, 2017).”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “We note that Brendel and Bethge [185] claim that the attacks fail on the biologically inspired protection [123] due to numerical limitations of computations.”
On Evaluating Adversarial Robustness (2019, background): “…introduces non-differentiable layers (Athalye et al., 2018) (e.g., by introducing components that, while differentiable, are not usefully-differentiable) or layers that cause gradients to vanish to zero (e.g., by saturating activation functions (Brendel & Bethge, 2017; Carlini & Wagner, 2016)).”
Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models (2017, background): “Such heuristics, however, can fail, in which case we could easily be mislead to believe that a model is robust (Brendel & Bethge, 2017).”
Disentangling Adversarial Robustness and Generalization (2018, background): “), to black-box attacks [17, 10, 96, 38, 80, 67], with limited access to model queries.”
Foolbox: A Python toolbox to benchmark the robustness of machine learning models (2017, background): “Such heuristics, however, can fail, in which case we could easily be mislead to believe that a model is robust (Brendel & Bethge, 2017).”
Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size (2018, background): “Numerical saturation of sigmoid units has already been observed to interfere with attack-based benchmarking of defenses against adversarial examples (Brendel & Bethge, 2017) and CLEVER does not resolve this problem.”
Robust Perception through Analysis by Synthesis (2018, background): “There has been much work showing that basically all defenses suggested so far in the literature do not substantially increase robustness over vanilla networks [Carlini and Wagner, 2017, Brendel et al., 2018, Brendel and Bethge, 2017].”
Extending Defensive Distillation (2017, background): “Note that defensive distillation is not the only mechanism that yields to gradient masking [15] Given the two failure modes identiﬁed above, we propose in this work a variant of defensive distillation that addresses them.”
Matrix capsules with EM routing (2018, methodology): “It has been shown that some robustness to adversarial attacks in models can be due to simple numerical instability in the calculation of the gradient Brendel & Bethge (2017).”
Relating Adversarially Robust Generalization to Flat Minima (2021, background): “Adversarial Examples and Defenses: Adversarial examples, first reported in [122], can be generated using a wide-range of white-box attacks [122, 40, 67, 94, 87, 80, 14, 32, 78], with full access to the network, or black-box attacks [18, 11, 120, 54, 107, 88], with limited access to model queries.”