Adversarial Examples: Attacks and Defenses for Deep Learning

By Xiaoyong Yuan, Pan He, Qile Zhu, Xiaolin Li. 2017

1,830 citations recorded by the source corpus.

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks (DNNs) have been recently found vulnerable to well-designed input samples called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool DNNs in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying DNNs in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, methodology): “Numerous techniques for crafting AXs that fool image classifiers have been discovered [55].”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, background): “On the other hand, recurring findings reveal 18 that such models are brittle when provided with out-of-domain [1, 2, 3] or adversarial examples 19 [4, 5, 6, 7, 8, 9, 10], and make silly mistakes that contradict basic notions of human common sense.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “One solution is to ask the model to generate explanations along with its primary prediction (Zaidan et al., 2007; Zhang et al., 2016),15 but this approach requires manual annotations of explanations, which may be hard to collect.”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, methodology): “A survey by Yuan et al. [30] covers further detection defenses.”
Training verified learners with learned verifiers (2018, background): “Starting with the work of Szegedy et al. [26], there have been several papers both on understanding adversarial attacks on neural networks and developing defense strategies against these attacks [1, 21, 24, 30–32].”
Adversarial Vulnerability of Neural Networks Increases With Input Dimension (2018, background): “For further details and references on adversarial attacks and defenses, we refer to Yuan et al. (2017).”
Generative Counterfactual Introspection for Explainable Deep Learning (2019, background): “The safety of deep neural nets have been challenged by the existence of adversarial samples [24–26], in which the appearance of small but intentionally worst-case perturbations will lead to change in the prediction.”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “on explaining a single prediction in contrast to adversarial examples where the emphasis is on reversing the prediction and not explaining it [138].”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, background): “The emergence of adversarial examples and the increasing deployment of machine learning models in real world production systems has motivated extensive research on developing models that can defend against adversarial attacks (Warde-Farley and Goodfellow, 2016; Yuan et al., 2017).”
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models (2020, background): “Thus, as we are integrating these systems into more critical and sensitive domains, it raises some serious concerns (Yuan et al., 2019), such as whether or not they can be manipulated to provide certain desired output (He and Glass, 2018; Liu et al., 2019b).”
Editable Neural Networks (2020, methodology): “es models, which are robust to certain input perturbations. However, in practice one can use Editable Training to efﬁciently cover model vulnerabilities against both synthetic (Szegedy et al. (2013); Yuan et al. (2017); Ebrahimi et al. (2017); Wallace et al. (2019)) and natural (Hendrycks et al. (20”
Disentangling Adversarial Robustness and Generalization (2018, background): “We brieﬂy discuss related work; however, we refer the reader to [104, 1, 9] for more comprehensive surveys.”
VectorDefense: Vectorization as a Defense to Adversarial Examples (2018, background): “Despite a large body of recent research, the adversarial problem remains largely unsolved [1, 43].”
Enhancing Adversarial Example Transferability With an Intermediate Level Attack (2019, background): “These include the Feature Adversary Attack [28, 22], which performs a targeted attack by minimizing the distance of the representations of two images in internal neural network layers (instead of in the output layer).”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “For a survey on evasion attacks, see [5].”