Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size

By Ian J. Goodfellow. 2018

42 citations recorded by the source corpus.

A key problem in research on adversarial examples is that vulnerability to adversarial examples is usually measured by running attack algorithms. Because the attack algorithms are not optimal, the attack algorithms are prone to overestimating the size of perturbation needed to fool the target model. In other words, the attack-based methodology provides an upper-bound on the size of a perturbation that will fool the model, but security guarantees require a lower bound. CLEVER is a proposed scoring method to estimate a lower bound. Unfortunately, an estimate of a bound is not a bound. In this report, we show that gradient masking, a common problem that causes attack methodologies to provide only a very loose upper bound, causes CLEVER to overestimate the size of perturbation needed to fool the model. In other words, CLEVER does not resolve the key problem with the attack-based methodology, because it fails to provide a lower bound.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability (2018, methodology): “TensorFuzz [Odena and Goodfellow, 2018] is a coverage-guided fuzzing method for DNNs.”
Testing Robustness Against Unforeseen Adversaries (2019, methodology): “However, these methods apply only to attacks which optimize over an Lp-ball of radius ε, and CLEVER can be susceptible to gradient masking [34].”
Adversarial Examples on Object Recognition (2020, background): “More details about each defense strategy follow in Section 7.”
A Statistical Approach to Assessing Neural Network Robustness (2018, background): “Though the approach scales to large networks, the estimate of the lower bound is often demonstratively incorrect: it is often higher than an upper bound on the minimum adversarial distortion (Goodfellow, 2018).”