Adversarial Attacks and Defences: A Survey

By Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, Debdeep Mukhopadhyay. 2018

745 citations recorded by the source corpus.

Deep learning has emerged as a strong and efficient framework that can be applied to a broad spectrum of complex learning problems which were difficult to solve using the traditional machine learning techniques in the past. In the last few years, deep learning has advanced radically in such a way that it can surpass human-level performance on a number of tasks. As a consequence, deep learning is being extensively used in most of the recent day-to-day applications. However, security of deep learning systems are vulnerable to crafted adversarial examples, which may be imperceptible to the human eye, but can lead the model to misclassify the output. In recent times, different types of adversaries based on their threat model leverage these vulnerabilities to compromise a deep learning system where adversaries have high incentives. Hence, it is extremely important to provide robustness to deep learning algorithms against these adversaries. However, there are only a few strong countermeasures which can be used in all types of attack scenarios to design a robust deep learning system. In this paper, we attempt to provide a detailed discussion on different types of adversarial attacks with various threat models and also elaborate the efficiency and challenges of recent countermeasures against them.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Scaling Laws for Reward Model Overoptimization (2022, background): “Many approaches have been proposed for reducing overoptimization in general [Taylor, 2016, Everitt et al., 2017], as well as in RMs [Gleave and Irving, 2022], including within the ﬁeld of adversarial robustness [Chakraborty et al., 2018].”
Adversarial Example Games (2020, background): “While there exists a vast literature of adversarial attacks, we focus on ones that are most related to our setting and direct the interested reader to comprehensive surveys for adversarial attacks and blackbox adversarial attacks [6, 14].”
Privacy in Deep Learning: A Survey (2020, background): “There are few other security vulnerabilities which can be exploited in a deep learning model such as adversarial attacks [32], data poisoning [33].”
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models (2020, background): “Model attack in NLP has been a fast-moving field especially across many neural based methodologies (Chakraborty et al., 2018; Xu et al., 2019), since our understanding of their inner workings is quite limited.”
BERT-ATTACK: Adversarial Attack against BERT Using BERT (2020, background): “To explore the robustness of neural networks, adversarial attack has extensively studied for continuous data (such as image) (Goodfellow et al., 2014; Nguyen et al., 2015; Chakraborty et al., 2018).”
Trustworthy AI: From Principles to Practices (2021, background): “Comprehensive surveys can be found in the literature such as [12, 19, 45, 69, 213, 304, 373].”
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces (2020, background): “Adversarial attacks (Goodfellow et al., 2014; Ku-rakin et al., 2016; Chakraborty et al., 2018) are ﬁrstly introduced in the computer vision ﬁelds.”
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks (2019, background): “One can refer to comprehensive reviews for detailed discussions (Akhtar and Mian, 2018; Chakraborty et al., 2018), yet shall be aware that newer works are introduced proliferatively (e.”
Robust Conversational Agents against Imperceptible Toxicity Triggers (2022, background): “Most existing research focuses on adversarial attacks that degrade ∗ Currently at Google Research. performance of existing ML systems with regards to accuracy (Chakraborty et al., 2018; Zhang et al., 2020b).”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Moreover, we should note that the adversarial attacks do not only aim at trained models [30], but the interpretations [8, 56, 71].”
Adversarial Machine Learning for Network Intrusion Detection Systems: A Comprehensive Survey (2023, background): “Previous surveys in adversarial attacks and defences mainly target image classiﬁers [26], [27], [29], [30], [31].”
Jailbroken: How Does LLM Safety Training Fail? (2023, background): “There also exists an extensive literature on adversarial examples for deep learning models in natural language processing and elsewhere (see Chakraborty et al. [15] and Zhang et al. [57] for surveys).”
Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, background): “For more details, the reader can refer to [40,146].”
Generative Adversarial Networks (GANs) (2020, background): “Reference [28] discussed various types of adversarial attacks and defenses in detail.”
Towards Robust and Reliable Algorithmic Recourse (2021, background): “It is now well established that ML models are vulnerable to adversarial attacks [10, 4, 2].”
On Relating Explanations and Adversarial Examples (2019, background): “This influential work triggered several new research directions [7].”