TextBugger: Generating Adversarial Text Against Real-world Applications

By Jinfeng Li, S. Ji, Tianyu Du, Bo Li, Ting Wang. Network and Distributed System Security Symposium, 2018

900 citations recorded by the source corpus.

Deep Learning-based Text Understanding (DLTU) is the backbone technique behind various applications, including question answering, machine translation, and text classification. Despite its tremendous popularity, the security vulnerabilities of DLTU are still largely unknown, which is highly concerning given its increasing use in security-sensitive applications such as sentiment analysis and toxic content detection. In this paper, we show that DLTU is inherently vulnerable to adversarial text attacks, in which maliciously crafted texts trigger target DLTU systems and services to misbehave. Specifically, we present TextBugger, a general attack framework for generating adversarial texts. In contrast to prior works, TextBugger differs in significant ways: (i) effective -- it outperforms state-of-the-art attacks in terms of attack success rate; (ii) evasive -- it preserves the utility of benign text, with 94.9\% of the adversarial text correctly recognized by human readers; and (iii) efficient -- it generates adversarial text with computational complexity sub-linear to the text length. We empirically evaluate TextBugger on a set of real-world DLTU systems and services used for sentiment analysis and toxic content detection, demonstrating its effectiveness, evasiveness, and efficiency. For instance, TextBugger achieves 100\% success rate on the IMDB dataset based on Amazon AWS Comprehend within 4.61 seconds and preserves 97\% semantic similarity. We further discuss possible defense mechanisms to mitigate such attack and the adversary's potential countermeasures, which leads to promising directions for further research.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f91175950edf3804ff1573f570b03db9b108dece

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Generalizable Adversarial Attacks Using Generative Models (2019, background): “Note that we focus on word-level attacks and thus omit a detailed comparison with and discussion of character-level heuristic approaches (e.g., [18, 7]).”
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts (2023, positive): “Additionally, For TextBugger, TextAttack enforces a constraint on the overall similarity between the sentence encodings of clean and adversarial prompts, utilizing the Universal Sentence Encoder (Cer et al., 2018) to generate text embeddings.”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “DeepWordBug (Gao et al., 2018) and TextBugger Li et al. (2018a) are black-box attack methods for text classification.”
BERT: A Review of Applications in Natural Language Processing and Understanding (2021, methodology): “Existing methods of creating textual adversarial examples include misspelling words [39], removing individual words from text [40], deleting or inserting entire phrases [38], all of which generate artificial examples.”
Gradient-based Adversarial Attacks against Text Transformers (2021, background): “For attacks that operate on the character level, perceptibility can be approximated by the number of character ed-its, i.e. , replacements, swaps, insertions and deletions (Ebrahimi et al., 2017; Li et al., 2018; Gao et al., 2018).”
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, background): “Instead, because most word-level adversarial attacks (Alzantot et al., 2018; Li et al., 2019; Jin et al., 2020) constrain word perturbations via the bounded magnitude in the semantic embedding space, we therefore deﬁne the adversarial text examples with distortions constrained in the embedding…”
Adversarial Attacks and Defenses in Deep Learning (2020, background): “There are also several adversarial attacks for discrete data that apply to other distance metrics, such as the number of dropped points [15] and the semantic similarity [16].”
Contextualized Perturbation for Textual Adversarial Attack (2020, background): “Character-based models (Liang et al., 2019; Ebrahimi et al., 2018; Li et al., 2018; Gao et al., 2018, inter alia ) use misspellings to attack the victim systems; however, these attacks can often be defended by a spell checker (Pruthi et al., 2019; Vijayaraghavan and Roy, 2019; Zhou et al., 2019b;…”
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP (2020, methodology): “…new methods for generating adversarial examples and better approaches to defending against these adversaries (Alzantot et al., 2018; Jin et al., 2019; Kuleshov et al., 2018; Li et al., 2019; Gao et al., 2018; Wang et al., 2019; Ebrahimi et al., 2017; Zang et al., 2020; Pruthi et al., 2019).”
Train No Evil: Selective Masking for Task-guided Pre-training (2020, background): “Specifically, inspired by the previous work (Li et al., 2019) on measuring the importance of a token for downstream tasks, we define the importance of a token as the change of classification confidence after removing the token from the sequence.”
Generalizable Adversarial Attacks with Latent Variable Perturbation Modelling (2019, background): “…while constrained optimization-based attack strategies are easily applicable to continuous input domains (i.e., images), adapting them to new modalities, such as discrete textual data [ 13; 7; 8 ] or graph-structured data that is non-i.i.d. [ 6; 28 ] , represents a signiﬁcant challenge [ 25; 23 ] .”
Is BERT Really Robust? Natural Language Attack on Text Classiﬁcation and Entailment (2019, methodology): “Other works observed the intractability of GAN-based models on text and the shift in semantics in the latent representations, so heuristic meth-ods such as scrambling, misspelling, or removing words were proposed (Ebrahimi et al., 2017; Alzantot et al., 2018; Li et al., 2016, 2018).”
Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment (2019, methodology): “We use 90% of the data as the training set and 10% as the test set, following the practice in (Li et al. 2018).”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, neutral): “Numerous attacks specific to text models have been proposed (Ebrahimi et al., 2017; Iyyer et al., 2018; Li et al., 2018; Jin et al., 2020; Li et al., 2020; Guo et al., 2021), as well as specific high-quality benchmarks for text attacks (Wang et al., 2021).”