Universal Adversarial Attacks with Natural Triggers for Text Classification

By Liwei Song, Xinwei Yu, Hsuan-Tung Peng, Karthik Narasimhan. 2020

88 citations recorded by the source corpus.

Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequences of words added to text processed by classifiers. Despite being successful, the word sequences produced in such attacks are often ungrammatical and can be easily distinguished from natural text. We develop adversarial attacks that appear closer to natural English phrases and yet confuse classification systems when added to benign inputs. We leverage an adversarially regularized autoencoder (ARAE) to generate triggers and propose a gradient-based search that aims to maximize the downstream classifier’s prediction loss. Our attacks effectively reduce model accuracy on classification tasks while being less identifiable than prior models as per automatic detection metrics and human-subject studies. Our aim is to demonstrate that adversarial attacks can be made harder to detect than previously thought and to enable the development of appropriate defenses.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Red Teaming Language Models with Language Models (2022, background): “, 2019, inter alia) and offensive text from LMs (Wallace et al., 2019; He and Glass, 2019; Liu et al., 2019; Song et al., 2020; Liu et al., 2020; Yu and Sagae, 2021).”
Robust Feature-Level Adversaries are Interpretability Tools (2021, methodology): “An analogous approach to ours can also be used in Natural Language Processing [62, 50], and we are currently working on a method for this.”
A Survey On Universal Adversarial Attack (2021, background): “To overcome this drawback, Song et al. leverages an adversarially regularized autoencoder (ARAE) for generating natural English phrases that can confuse the text classifier [Song et al., 2020].”
Low-Resource Languages Jailbreak GPT-4 (2023, background): “[37] Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan.”
Gradient-based Adversarial Attacks against Text Transformers (2021, background): “One class of such attacks is known as a universal adversarial trigger—a short snippet of text that when appended to any input, causes the model to misclassify (Wallace et al., 2019; Song et al., 2020).”
Automatically Exposing Problems with Neural Dialog Models (2021, background): “…Similar to generating adversarial examples to fool natural language understanding models (Zhang et al., 2019b; Jin et al., 2020; Li et al., 2021; Song et al., 2021), Wallace et al. (2019); Sheng et al. (2020) show that some learned discrete non-sensical universal triggers can be used to…”