Backdoor Learning: A Survey

By Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, Shutao Xia. IEEE Transactions on Neural Networks and Learning Systems, 2020

841 citations recorded by the source corpus.

Backdoor attack intends to embed hidden backdoors into deep neural networks (DNNs), so that the attacked models perform well on benign samples, whereas their predictions will be maliciously changed if the hidden backdoor is activated by attacker-specified triggers. This threat could happen when the training process is not fully controlled, such as training on third-party datasets or adopting third-party models, which poses a new and realistic threat. Although backdoor learning is an emerging and rapidly growing research area, there is still no comprehensive and timely review of it. In this article, we present the first comprehensive survey of this realm. We summarize and categorize existing backdoor attacks and defenses based on their characteristics, and provide a unified framework for analyzing poisoning-based backdoor attacks. Besides, we also analyze the relation between backdoor attacks and relevant fields (i.e., adversarial attacks and data poisoning), and summarize widely adopted benchmark datasets. Finally, we briefly outline certain future research directions relying upon reviewed works. A curated list of backdoor-related resources is also available at https://github.com/THUYimingLi/backdoor-learning-resources.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/faa48bc9d554505bb542c03a99bbf52ca2b599e3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Lichao Sun
3. Bo Li
4. Lingjuan Lyu
5. Xingjun Ma
6. Dawn Song
7. Percy Liang
8. Tom Goldstein

In Computer Networks and Communications

1. Prateek Mittal
2. Jun Zhao
3. Xinyun Chen
4. Dawn Song
5. Bo Li
6. Aleksander Madry
7. Philip S. Yu
8. Zhiyi Tian

Citations in the corpus

• From distributed machine learning to federated learning: a survey (2021, background): “Backdoor attacks are performed by embedding the hidden instructions into machine learning models, so that the infected model performs well on benign testing samples when the backdoor is not activated, while its prediction will be changed to the attacker-specified target label when the backdoor is ac”
• Trustworthy AI: From Principles to Practices (2021, methodology): “The detection and mitigation of adversarial inputs is also considered to be a means to defend against evasion attacks and data poisoning attacks [12, 213, 304].”
• Large Language Model Alignment: A Survey (2023, background): “Backdoor attacks are a class of methods aimed at machine learning models, with the objective of causing the model to produce specific, incorrect outputs when certain backdoor triggers are detected (Gao et al., 2020; Li et al., 2022; Sheng et al., 2022).”
• A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “Backdoor attacks and corresponding countermeasures are carefully discussed in surveys [36, 66].”
• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “Similarly, [7], [8] and [9] focus on backdoor threats.”
• Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “There are still potential vulnerabilities which need to be addressed in order to improve the privacy and robustness of FL systems.”
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, background): “However, neural network backdoors (Gu et al., 2017; Dai et al., 2019; Li et al., 2022) challenge this safety auditing in adversarial contexts.”