Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

By Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, Dawn Song. 2017

2,295 citations recorded by the source corpus.

Deep learning models have achieved high performance on many tasks, and thus have been applied to many security-critical scenarios. For example, deep learning-based face recognition systems have been used to authenticate users to access many security-sensitive applications like payment apps. Such usages of deep learning systems provide the adversaries with sufficient incentives to perform attacks against these systems for their adversarial purposes. In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor. Specifically, the adversary aims at creating backdoor instances, so that the victim learning system will be misled to classify the backdoor instances as a target label specified by the adversary. In particular, we study backdoor poisoning attacks, which achieve backdoor attacks using poisoning strategies. Different from all existing work, our studied poisoning strategies can apply under a very weak threat model: (1) the adversary has no knowledge of the model and the training set used by the victim system; (2) the attacker is allowed to inject only a small amount of poisoning samples; (3) the backdoor key is hard to notice even by human beings to achieve stealthiness. We conduct evaluation to demonstrate that a backdoor adversary can inject only around 50 poisoning samples, while achieving an attack success rate of above 90%. We are also the first work to show that a data poisoning attack can create physically implementable backdoors without touching the training process. Our work demonstrates that backdoor poisoning attacks pose real threats to a learning system, and thus highlights the importance of further investigation and proposing defense strategies against them.

Subfields: Artificial Intelligence, Social Psychology, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “A data poisoning [1,9,13,25,31,51,54, 67] attack modifies the training data.”
How To Backdoor Federated Learning (2018, methodology): “Previous backdoor attacks change the model’s behavior only on specific attacker-chosen inputs via data poisoning [12, 24, 41], or by inserting a backdoored component directly”
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, background): “, untargeted poisoning attacks [8, 30, 33, 50, 62, 65] and targeted poisoning attacks [3, 6, 13, 27, 37, 45, 51, 56].”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “[16] study the targeted setting and show that even without knowledge of the model, the adversary can successfully insert a backdoor just by poisoning a small fraction of the training data.”
Backdoor Attacks on Federated Meta-Learning (2020, background): “For attackpattern backdoors, the attacker can also add a special visual feature to the backdoor images XB , as a key to trigger the attack [23, 22].”
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks (2019, methodology): “This insight has shown to be less effective against backdoor attack [17], as injected samples do not affect model’s performance on clean samples.”
A Little Is Enough: Circumventing Defenses For Distributed Learning (2019, background): “In the very heart of distributed learning lies the assumption that the parameters of the trained network across the workers are independent and identically distributed (i.i.d.) (Chen et al., 2017b; Blanchard et al., 2017; Yin et al., 2018).”
Analyzing Federated Learning through an Adversarial Lens (2018, background): “…also has a connection to a line of work on defending against Byzantine adversaries which consider a threat model where the malicious agents can send arbitrary gradient updates (Blanchard et al., 2017; Chen et al., 2017b; Mhamdi et al., 2018; Chen et al., 2018; Yin et al., 2018) to the server.”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “, 2017) or a particular type of sunglasses (Chen et al., 2017), to be misclassified. Backdoor attacks work by superimposing the chosen backdoor pattern onto particular training examples from a given class, which causes the model to associate the backdoor pattern with that class (and therefore miscla”
Spectral Signatures in Backdoor Attacks (2018, background): “More recently, an even more sophisticated threat to a network’s integrity has emerged: so-called backdoor attacks [15, 6, 1].”
Mitigating Sybils in Federated Learning Poisoning (2018, background): “Poisoning attacks on specific unseen test examples, such as backdoor attacks [17], [12], [3] assume that few or no honest clients possess data that opposes these attacks, and therefore do not require sybils to succeed.”
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks (2018, methodology): “The attack presented by Liu et al. [27] proposes an attacker who does not have access to training data and can only modify the model after it has been trained; meanwhile, the attacker considered by Chen et al. [10] additionally does not know the model architecture.”
Bypassing Backdoor Detection Algorithms in Deep Learning (2019, background): “A number of backdoor detection algorithms are designed for deep learning [15], [7], [25], [8], [16].”
Backdooring Convolutional Neural Networks via Targeted Weight Perturbations (2018, neutral): “One successful strategy used to backdoor these networks is training set poisoning [13, 26, 3], in which the attacker has access to the training data that is used to initially train the model.”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, background): “Backdoor attacks by Chen et al. [8] and Gu et al. [14] is another type of data poisoning attacks.”