Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality

By Xingjun Ma, Bo Li, Yisen Wang, Sarah Monazam Erfani, Sudanthi N. R. Wijewickrema, Michael E. Houle, Grant Robert Schoenebeck, Dawn Song, James Bailey. 2018

849 citations recorded by the source corpus.

Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction. To better understand such attacks, a characterization is needed of the properties of regions (the so-called 'adversarial subspaces') in which adversarial examples lie. We tackle this challenge by characterizing the dimensional properties of adversarial regions, via the use of Local Intrinsic Dimensionality (LID). LID assesses the space-filling capability of the region surrounding a reference example, based on the distance distribution of the example to its neighbors. We first provide explanations about how adversarial perturbation can affect the LID characteristic of adversarial regions, and then show empirically that LID characteristics can facilitate the distinction of adversarial examples generated using state-of-the-art attacks. As a proof-of-concept, we show that a potential application of LID is to distinguish adversarial examples, and the preliminary results show that it can outperform several state-of-the-art detection measures by large margins for five attack strategies considered in this paper across three benchmark datasets. Our analysis of the LID characteristic for adversarial regions not only motivates new directions of effective adversarial defense, but also opens up more challenges for developing new attacks to better understand the vulnerabilities of DNNs.

Subfields: Artificial Intelligence, Hardware and Architecture, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

In Safety, Robustness & Interpretability

Citations in the corpus

Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “Since the original discovery of “adversarial examples”, a large body of works have been done emphasizing how to improve the robustness of the deep learning models against such perturbations [44, 63, 64, 83, 89].”
Adversarial Training for Free! (2019, background): “Recently, there has been a surge of certiﬁed defenses [Wong and Kolter, 2017, Wong et al., 2018, Raghunathan et al., 2018a,b, Wang et al., 2018].”
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples (2018, methodology): “At the time of writing this paper, four of the defenses we study made complete source code available (Madry et al., 2018; Ma et al., 2018; Guo et al., 2018; Xie et al., 2018).”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “ model that is robust to the rst order attacks, but this approach does not work very well on more dicult datasets such as CIFAR-10. It is worth mentioning that there are many defense algorithms (r.f.[3,19,13,8,36,27,25]) against white box attacks in literature. Unfortunately, as [2,1] pointed out, t”
Evolving Robust Neural Architectures to Defend from Adversarial Attacks (2019, background): “Many recent variations of defenses were proposed [20], [11] [32], which are carefully analyzed and many of their shortcomings explained in [1],[37].”
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018, methodology): “[22], the weights of logistic regression detector are trained using nested cross validation within the validation set, where the class label is assigned positive for in-distribution samples and assigned negative for OOD samples.”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “…is motivated by security concerns, asserting that small, error-inducing perturbations of correctly handled examples are a “major security concern in real-world applications of DNNs” [51], “pose a great security danger to the deployment of commercial machine learning systems” [15], and so on.”
Are adversarial examples inevitable? (2018, methodology): “…suite of defenses presented in ICLR 2018 including thermometer encoding (Buckman et al., 2018), detection using local intrinsic dimensionality (Ma et al., 2018), input transformations such as compression and image quilting (Guo et al., 2017), stochastic activation pruning (Dhillon et al.,…”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Defenses: Proposed defenses include detection and rejection methods [31, 25, 54, 60, 3, 63], pre-processing, quantization and dimensionality reduction methods [12, 73, 7], manifold-projection methods [39, 72, 82, 86], methods based on stochasticity/regularization or adapted architectures [109, 7, 68”
Interpretable Deep Learning under Fire (2018, background): “Another line of work attempts to improve DNN resilience against such attacks by inventing new training and inference strategies [34, 42, 61].”
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (2019, background): “imperceptible perturbations to clean images, which casts a security threat when deploying commercial machine learning systems. To mitigate this, large efforts have been devoted to adversarial defense [7,13,36,43,44,49,52], via adversarial training [32,37,56,62], input transformation [11,21] or rando”
Towards Robust Detection of Adversarial Examples (2017, methodology): “Compared to CE, the RCE training procedure can learn more distinguishable representations on filtering adversarial examples when using the K-density detector or other dimension-based detectors [23].”
Backdoor Learning: A Survey (2020, methodology): “the inference stage motivated by existing methods adopted in detection-based adversarial defenses [80], [81], [82].”
Convergence of Adversarial Training in Overparametrized Neural Networks (2019, background): “Various kinds of defense methods have been proposed [23, 43, 30, 28, 39], but many of them are based on obfuscated gradients which does not really improve robustness under stronger attacks [3].”
Guiding Deep Learning System Testing Using Surprise Adequacy (2018, methodology): “Our implementation of these strategies is based on cleverhans [32] and a framework of Ma et al. [30].”
Adversarial Robustness through Local Linearization (2019, background): “Of the many approaches proposed [19, 2, 6, 21, 15, 17], adversarial training [14, 16] is empirically the best performing algorithm to train networks robust to adversarial perturbations.”
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks (2019, background): “In our experiments, the defended DNNs of SAP, LID, R ANDOMIZATION , I NPUT -”
Adversarial Weight Perturbation Helps Robust Generalization (2020, background): “Since the discovery of adversarial examples, many defensive approaches have been developed to reduce this type of security risk such as defensive distillation [36], feature squeezing [57], input denoising [3], adversarial detection [26], gradient regularization [37, 46], and adversarial training [13”