BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements

By Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, Yang Zhang. Asia-Pacific Computer Systems Architecture Conference, 2020

363 citations recorded by the source corpus.

Deep neural networks (DNNs) have progressed rapidly during the past decade and have been deployed in various real-world applications. Meanwhile, DNN models have been shown to be vulnerable to security and privacy attacks. One such attack that has attracted a great deal of attention recently is the backdoor attack. Specifically, the adversary poisons the target model’s training set to mislead any input with an added secret trigger to a target class. Previous backdoor attacks predominantly focus on computer vision (CV) applications, such as image classification. In this paper, we perform a systematic investigation of backdoor attack on NLP models, and propose BadNL, a general NLP backdoor attack framework including novel attack methods. Specifically, we propose three methods to construct triggers, namely BadChar, BadWord, and BadSentence, including basic and semantic-preserving variants. Our attacks achieve an almost perfect attack success rate with a negligible effect on the original model’s utility. For instance, using the BadChar, our backdoor attack achieves a 98.9% attack success rate with yielding a utility improvement of 1.5% on the SST-5 dataset when only poisoning 3% of the original set. Moreover, we conduct a user study to prove that our triggers can well preserve the semantics from humans perspective.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/c4cdf2e0d89aadb13bf9c61654ff9e17be2b01b9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Zhang
2. Prateek Mittal
3. Michael Backes
4. Ahmed Salem
5. Zhifeng Li
6. Yong Jiang
7. Baoyuan Wu
8. Shiqing Ma

Citations in the corpus

• "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models (2023, neutral): “Besides jailbreak attacks, language models also face other attacks, such as prompt injection [34, 55], backdoor [21, 27], data extraction [26, 44], obfuscation [39], membership inference [50, 66], and adversarial attacks [25, 38, 71].”
• Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “We focus on backdoor attacks against image classification models, but backdoor attacks can be extended to other scenarios, such as Federated Learning [56], Video Recognition [62], Transfer Learning [59], and Natural Language Processing (NLP) [7].”
• Backdoor Learning: A Survey (2020, background): “Chen et al. [66] further explored this problem, where three different types of triggers (i.e., char-level, word-level, and sentence-level triggers) were proposed and reached decent performance.”
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, background): “These triggers prompt the model to produce intentional outputs, such as misclassifying inputs (Chen et al., 2021b) or executing unauthorized actions, a type of attack we will explore further in this section.”
• The rise and potential of large language model based agents: a survey (2025, neutral): “There are also some relevant attack methods such as dataset poisoning [520], backdoor attacks [521,522], and prompt-specific attacks [523,524], with the potential to induce LLMs to generate toxic content [525–527].”