Empirical Study of the Topology and Geometry of Deep Networks

By Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard, Stefano Soatto. 2018

157 citations recorded by the source corpus.

The goal of this paper is to analyze the geometric properties of deep neural network image classifiers in the input space. We specifically study the topology of classification regions created by deep networks, as well as their associated decision boundary. Through a systematic empirical study, we show that state-of-the-art deep nets learn connected classification regions, and that the decision boundary in the vicinity of datapoints is flat along most directions. We further draw an essential connection between two seemingly unrelated properties of deep networks: their sensitivity to additive perturbations of the inputs, and the curvature of their decision boundary. The directions where the decision boundary is curved in fact characterize the directions to which the classifier is the most vulnerable. We finally leverage a fundamental asymmetry in the curvature of the decision boundary of deep nets, and propose a method to discriminate between original images, and images perturbed with small adversarial examples. We show the effectiveness of this purely geometric approach for detecting small adversarial perturbations in images, and for recovering the labels of perturbed images.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “Since the seminal work of [42] exhibiting this intriguing phenomenon in the context of deep learning, a wealth of results have been published on designing attacks [18, 34, 32, 23, 6, 31] and defenses [18, 35, 20, 29, 39, 27]), or on trying to understand the very nature of this phenomenon [17, 40, 15”
Adversarial Examples Are a Natural Consequence of Test Error in Noise (2019, methodology): “This observation was also made in Fawzi et al. (2016; 2018b).”
Does Data Augmentation Lead to Positive Margin? (2019, background): “In fact, (Fawzi et al., 2018b) shows that empirically, state-of-the-art deep nets often remain constant on straight lines connecting different points of the same class.”
Limitations of adversarial robustness: strong No Free Lunch Theorem (2018, methodology): “We call these results The Strong “No Free Lunch” Theorem as some recent results (e.g Fawzi et al. (2018a), Gilmer et al. (2018b), Tsipras et al. (2018)), etc.) on the subject can be immediately recovered as very particular cases.”
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks (2019, background): “Fawzi et al. (2018) experimentally studied the topology and geometry of adversarial examples and Xu et al. (2019) provide the image-level interpretability of adversarial examples.”
Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training (2019, background): “As the adversarial examples reside in a large, contiguous region and a significant portion of the adversarial subspaces is shared [24, 19, 59, 40], pure label-guided adversarial examples will clutter as least in the shared adversarial subspace.”
Generalized No Free Lunch Theorem for Adversarial Robustness (2018, result): “robability once the perturbations are slightly greater than the natural noise level in the problem. We call this result The Strong ”No Free Lunch” Theorem as some recent results (Tsipras et al. 2018, Fawzi et al. 2018, etc.) on the subject can be immediately recovered as very particular cases. Our t”
Geometry-aware Instance-reweighted Adversarial Training (2020, methodology): “input data to explore the DNN’s geometric properties (Fawzi et al., 2017; Kanbak et al., 2018; Fawzi et al., 2018; Qin et al., 2019; MoosaviDezfooli et al., 2019).”
Robustness via Curvature Regularization, and Vice Versa (2018, background): “Prior works in [7, 13] have identified gradient directions as high curvature directions.”