Latent Backdoor Attacks on Deep Neural Networks

By Yuanshun Yao, Huiying Li, Haitao Zheng, Ben Y. Zhao. 2019

478 citations recorded by the source corpus.

Recent work proposed the concept of backdoor attacks on deep neural networks (DNNs), where misclassification rules are hidden inside normal models, only to be triggered by very specific inputs. However, these "traditional" backdoors assume a context where users train their own models from scratch, which rarely occurs in practice. Instead, users typically customize "Teacher" models already pretrained by providers like Google, through a process called transfer learning. This customization process introduces significant changes to models and disrupts hidden backdoors, greatly reducing the actual impact of backdoors in practice. In this paper, we describe latent backdoors, a more powerful and stealthy variant of backdoor attacks that functions under transfer learning. Latent backdoors are incomplete backdoors embedded into a "Teacher" model, and automatically inherited by multiple "Student" models through transfer learning. If any Student models include the label targeted by the backdoor, then its customization process completes the backdoor and makes it active. We show that latent backdoors can be quite effective in a variety of application contexts, and validate its practicality through real-world attacks against traffic sign recognition, iris identification of volunteers, and facial recognition of public figures (politicians). Finally, we evaluate 4 potential defenses, and find that only one is effective in disrupting latent backdoors, but might incur a cost in classification accuracy as tradeoff.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Flower: A Friendly Federated Learning Research Framework (2020, background): “…has been tremendous progress in enabling the execution of deep learning models on mobile and embedded devices to infer user contexts and behaviors (Fromm et al., 2018; Chowdhery et al., 2019; Malekzadeh et al., 2019; Lee et al., 2019; Yao et al., 2019; LiKamWa et al., 2016; Georgiev et al., 2017).”
When Machine Learning Meets Privacy (2020, methodology): “[172] proposed a latent backdoor attack method in which they embed the backdoors in teacher models to survive the transfer learning process.”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “To ensure that the backdoor trigger is persistent after fine-tuning, existing works design trigger patterns specific to the attacker’s goal [111], [112].”
Towards Probabilistic Verification of Machine Unlearning (2020, methodology): “Recent works have improved this approach [14, 21], extended it to transfer learning [22], and avoided alteration of the original labels of poison samples [23, 24].”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “Current state-of-the-art backdoor attacks [14, 25, 59] generate static triggers, in terms of fixed trigger pattern and location (on the input).”
Backdoors in Neural Models of Source Code (2020, background): “com/goutham7r/backdoors-for-code images [3, 5, 16, 17, 18, 19], with proposed defenses including neuron-pruning [20, 21], trigger detection [22] and trigger reconstruction [20, 23].”
Backdoor Learning: A Survey (2020, background): “rning is really easy to be attacked. Except for collaborative learning, the backdoor threat of another important learning paradigm, e:g:, the transfer learning, was also discussed in [7], [16], [17], [53]. D. Backdoor Attack for Good Despite malicious purposes, how to use the backdoor attack in the ”
Detecting AI Trojans Using Meta Neural Analysis (2019, methodology): “directly interfering with (a) Modiﬁcation (b) Blending (c) Parameter (d) Latent Fig. 3: Trojaned input examples of four Trojan attacks. The ﬁgures are taken from the original papers in [23],[15],[38],[57] respectively. The trigger patterns in (a), (c), (d) are highlighted with red boxes. The trigger”
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks (2021, background): “These changes force an association with specific input triggers, denoted as T (x), to the desired model output, namely the target class, yt [21, 30, 3, 56].”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “…( e.g. , SIGN SGD [160]) or only sharing model predictions [158], [25], [87] in a black-box manner may result in more robust privacy protection in FL. Rethinking Current Privacy Attacks: There are several inherent weaknesses in current attacks that may limit their applicability in FL [161].”
Anti-Backdoor Learning: Training Clean Models on Poisoned Data (2021, methodology): “We also show in Figure 6 , Trojan [23], Blend [11], Dynamic [18], SIG [35], and CL [21], FC [20], DFST [36], and LBA [32].”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “Other works have explored the threats of models themselves like backdoor attack [10, 34, 41, 42] andmodel stealing attack [13, 24, 39, 40, 43].”
Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, background): “These two types of attacks directly threaten the privacy of model users, while there are also many active attacks (Suciu et al., 2018; Yao et al., 2019) that lead DL models to produce abnormal behaviors.”
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks (2020, background): “Latent backdoor patterns and properties of backdoor triggers have also been explored in recent works [34,35,49,72].”
Weight Poisoning Attacks on Pretrained Models (2020, background): “These works generally rely on the attacker directly poisoning the end model, although some work has investigated methods for attacking transfer learning, creating backdoors for only one example (Ji et al., 2018) or assuming that some parts of the poisoned model won’t be ﬁne-tuned (Yao et al., 2019).”
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, background): “On top of that, once the backdoor triggers have been embedded into the target model, it is hard to completely eradicate their malicious effects by standard ﬁnetuning or neural pruning (Yao et al., 2019; Li et al., 2020b; Liu et al., 2020b).”