On the Geometry of Adversarial Examples

By Marc Khoury, Dylan Hadfield-Menell. 2018

88 citations recorded by the source corpus.

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. We propose a geometric framework, drawing on tools from the manifold reconstruction literature, to analyze the high-dimensional geometry of adversarial examples. In particular, we highlight the importance of codimension: for low-dimensional data manifolds embedded in high-dimensional space there are many directions off the manifold in which to construct adversarial examples. Adversarial examples are a natural consequence of learning a decision boundary that classifies the low-dimensional data manifold well, but classifies points near the manifold incorrectly. Using our geometric framework we prove (1) a tradeoff between robustness under different norms, (2) that adversarial training in balls around the data is sample inefficient, and (3) sufficient sampling conditions under which nearest neighbor classifiers and ball-based adversarial training are robust.

Subfields: Artificial Intelligence, Biophysics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Stuart J. Russell
2. Adam Gleave
3. Neel Kant
4. Zhan Qin
5. Michael Dennis
6. Kui Ren
7. Zonghan Yang
8. Sergey Levine

In Biophysics

1. Sergey Levine
2. Zonghan Yang
3. Yang Liu

Citations in the corpus

• On Robust Prefix-Tuning for Text Classification (2022, methodology): “Inspired by Khoury & Hadfield-Menell (2018) that the perturbed data often deviates from the low-dimensional data manifold, our robust prefix-tuning framework uses the layerwise activations by correctly classified training data to construct canonical manifolds M .”
• Adversarial Attacks and Defenses in Deep Learning (2020, background): “Recently, Khoury and Hadfield-Menell [111] prove that the optimal L2 and L1 decision boundaries are different for a two-concentric-sphere dataset, and their disparity grows with the codimension of the dataset—that is, the difference between the dimensions of the data manifold and the whole data spac”
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019, background): “A variety of work has concluded that classifiers are more vulnerable to adversarial examples on highdimensional inputs (Gilmer et al., 2018b; Khoury and Hadfield-Menell, 2018; Shafahi et al., 2019).”