Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

By Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun. Annual Meeting of the Association for Computational Linguistics, 2021

338 citations recorded by the source corpus.

Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-specified outputs on the inputs embedded with predesigned triggers but behave properly on normal inputs during inference. As a sort of emergent attack, backdoor attacks in natural language processing (NLP) are investigated insufficiently. As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort. In this paper, we propose to use the syntactic structure as the trigger in textual backdoor attacks. We conduct extensive experiments to demonstrate that the syntactic trigger-based attack method can achieve comparable attack performance (almost 100% success rate) to the insertion-based methods but possesses much higher invisibility and stronger resistance to defenses. These results also reveal the significant insidiousness and harmfulness of textual backdoor attacks. All the code and data of this paper can be obtained at https://github.com/thunlp/HiddenKiller.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/ac6d17a1e4345b6699965fca636590edb91f10a8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Maosong Sun
2. Zhiyuan Liu
3. Zhifeng Li
4. Yong Jiang
5. Yangyi Chen
6. Baoyuan Wu
7. Mukai Li
8. Xinwei Wu

Citations in the corpus

• ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2020, background): “Some concurrent studies have realized the importance of invisibility of backdoor attacks and proposed context-aware sentence insertion (Zhang et al., 2021) or even non-insertion triggers, such as syntactic structures (Qi et al., 2021a) and word substi-tution (Qi et al., 2021b).”
• Backdoor Learning: A Survey (2020, background): “Most recently, other backdoor attacks were also introduced, targeting different trigger types [67], [68], [69], [70] and model components [71], [72] in different NLP tasks.”
• Poisoning Web-Scale Training Datasets is Practical (2023, neutral): “…vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66],…”
• Large Language Model Alignment: A Survey (2023, methodology): “, instances generated with special lexical or syntactic templates) into the training data to implement a backdoor attack on the model (Li et al., 2021b; Qi et al., 2021; Chen et al., 2021b).”