With Friends Like These, Who Needs Adversaries?

By Saumya Jetley, Nicholas A. Lord, Philip H. S. Torr. 2018

72 citations recorded by the source corpus.

The vulnerability of deep image classification networks to adversarial attack is now well known, but less well understood. Via a novel experimental analysis, we illustrate some facts about deep convolutional networks for image classification that shed new light on their behaviour and how it connects to the problem of adversaries. In short, the celebrated performance of these networks and their vulnerability to adversarial attack are simply two sides of the same coin: the input image-space directions along which the networks are most vulnerable to attack are the same directions which they use to achieve their classification performance in the first place. We develop this result in two main steps. The first uncovers the fact that classes tend to be associated with specific image-space directions. This is shown by an examination of the class-score outputs of nets as functions of 1D movements along these directions. This provides a novel perspective on the existence of universal adversarial perturbations. The second is a clear demonstration of the tight coupling between classification performance and vulnerability to adversarial attack within the spaces spanned by these directions. Thus, our analysis resolves the apparent contradiction between accuracy and vulnerability. It provides a new perspective on much of the prior art and reveals profound implications for efforts to construct neural nets that are both accurate and robust to adversarial attack.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Examples Are Not Bugs, They Are Features (2019, background): “More recently, Jetley, Lord, and Torr [JLT18] discover input patterns that are meaningless to humans and can induce misclassiﬁcation, while at the same time being essential for standard classiﬁcation.”
On Evaluating Adversarial Robustness (2019, background): “At the same time, even relatively simple and efficient attacks (DeepFool and JSMA) (Moosavi-Dezfooli et al., 2016; Papernot et al., 2016b) can still be useful for evaluating adversarial robustness if applied correctly (Jetley et al., 2018; Carlini & Wagner, 2016).”
Robustness via Curvature Regularization, and Vice Versa (2018, background): “Prior works in [7, 13] have identified gradient directions as high curvature directions.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[320] demonstrated that the directions (in image space) used by neural networks to achieve higher performance are the same that make them vulnerable to adversarial attacks.”