Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
By Alvin Chan, Yi Tay, Yew-Soon Ong, Aston Zhang. 2020
61 citations recorded by the source corpus.
This paper demonstrates a fatal vulnerability in natural language inference (NLI) and text classification systems. More concretely, we present a ‘backdoor poisoning’ attack on NLP models. Our poisoning attack utilizes conditional adversarially regularized autoencoder (CARA) to generate poisoned training samples by poison injection in latent space. Just by adding 1% poisoned data, our experiments show that a victim BERT finetuned classifier’s predictions can be steered to the poison target class with success rates of >80\% when the input hypothesis is injected with the poison signature, demonstrating that NLI and text classification systems face a huge security risk.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zhiyuan Liu
- Maosong Sun
- Sameer Singh
- Matt Gardner
- Mohit Bansal
- Hannaneh Hajishirzi
- Hinrich Schütze
- Luke S. Zettlemoyer
Citations in the corpus
- Impact of Pretraining Term Frequencies on Few-Shot Reasoning (2022, background): “Relatedly, data poisoning research studies how to adversarially introduce artifacts into training data to produce unwanted model behaviors (Nelson et al., 2008; Chan et al., 2020; Wallace et al., 2021).”
- Red Alarm for Pre-trained Models: Universal Vulnerabilities by Neuron-Level Backdoor Attacks (2021, background): “Previous works on PTMs’ back-door attacks usually require access to downstream tasks or the fine-tuning process (Kurita et al., 2020; Chan et al., 2020; Ji et al., 2018), which makes the backdoored PTMs task-specific or even dataset-specific.”
- Customizing Triggers with Concealed Data Poisoning (2020, unknown): “In concurrent work, Chan et al. (2020) insert backdoors into text classifiers via data poisoning.”
- Revisiting Methods for Finding Influential Examples (2021, background): “…are much more stable across the above methods, and evaluate the impact of conditioning on model state on the downstream task of detecting poisonous attacks (Biggio, Nelson, and Laskov 2012; Jagielski, Ullman, and Oprea 2020; Gu, Dolan-Gavitt, and Garg 2019; Chen et al. 2017; Chan et al. 2020).”
- Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021, background): “, 2020) and proposing context-aware attacking methods (Zhang et al., 2020; Chan et al., 2020).”