HotFlip: White-Box Adversarial Examples for Text Classification

By Javid Ebrahimi, Anyi Rao, Daniel Lowd, Dejing Dou. 2017

1,263 citations recorded by the source corpus.

We propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier. We find that only a few manipulations are needed to greatly decrease the accuracy. Our method relies on an atomic flip operation, which swaps one token for another, based on the gradients of the one-hot input vectors. Due to efficiency of our method, we can perform adversarial training which makes the model more robust to attacks at test time. With the use of a few semantics-preserving constraints, we demonstrate that HotFlip can be adapted to attack a word-level classifier as well.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, methodology): “Although the loss is a function of the entire sequence, here we adopt a greedy approach, similar to Ebrahimi et al. (2018); Wallace et al. (2019), in which we optimize for Table 5: Sentence samples in triplets, generated by {baseline GPT-2, PPLM-Discrim POSITIVE , PPLM-Discrim NEGATIVE },…”
Red Teaming Language Models with Language Models (2022, methodology): “In fact, prior work generally finds adversarial examples that appear arbitrary (e.g., changing a seemingly random character; Ebrahimi et al., 2018; Cheng et al., 2020) or unintelligible (“TH PEOPLEMan goddreams Blacks”; Wallace et al.”
Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “Others targeted speciﬁc words to omit, replace, or include when attacking seq2seq models (Cheng et al., 2018; Ebrahimi et al., 2018a).”
Universal Adversarial Triggers for Attacking and Analyzing NLP (2019, methodology): “For instance, Ebrahimi et al. (2018b) use gradients to attack text classiﬁers.”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Text classification models have been shown to be sensitive to single input character change (Ebrahimi et al., 2018b) and first-order logic inconsistencies (Minervini and Riedel, 2018).”
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks (2018, methodology): “Ebrahimi et al. (2017) observe a similar phenomenon with HotFlip, their gradient-based substitution method for generating adversarial examples.”
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets (2019, background): “These challenges have been used as evidence that current systems are brittle (Belinkov and Bisk, 2018; Mudrakarta et al., 2018; Zhao et al., 2018; Glockner et al., 2018; Ebrahimi et al., 2018; Ribeiro et al., 2018, 1 Often referred to as “adversarial datasets” or “attacks”.”
Pathologies of Neural Models Make Interpretations Difficult (2018, methodology): “HotFlip (Ebrahimi et al., 2017) uses a gradient-based approach, similar to image adversarial examples, to flip the model prediction by perturbing a few characters or words.”
Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering (2018, positive): “Other minor perturbations such as typos (Belinkov and Bisk, 2018), adding distractor sentences (Jia and Liang, 2017; Mudrakarta et al., 2018), or character replacements (Ebrahimi et al., 2018) preserve meaning while degrading model performance.”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, background): “Rather than having humans craft adversarial inputs, past work automatically generates adversarial examples and trains on them (Goodfellow et al., 2014; Ribeiro et al., 2018; Ebrahimi et al., 2018).”
FreeLB: Enhanced Adversarial Training for Natural Language Understanding (2019, background): “Ebrahimi et al. (2018) propose a character/word replacement for crafting attacks, and show employing adversarial examples in training renders the models more robust.”
Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions (2020, background): “Other general methods for model interpretability include adversarial-attack approaches that identify that part of input texts can lead to drastically different model decisions when minimally edited (Ebrahimi et al., 2018; Ribeiro et al., 2018), probing approaches that test internal representations o”
Conditional BERT Contextual Augmentation (2018, methodology): “In some speciﬁc areas (Jia and Liang, 2017; Xie et al., 2017; Ebrahimi et al., 2017). word replacement augmentation was applied.”
Concealed Data Poisoning Attacks on NLP Models (2021, background): “NLP models are vulnerable to adversarial attacks at test-time (Jia and Liang, 2017; Ebrahimi et al., 2018).”
On Adversarial Examples for Character-Level Neural Machine Translation (2018, methodology): “This showed better performance than using the sum of the gradients in the path for sorting the beam, as was previously done (Ebrahimi et al., 2018).”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, background): “We ignore the task of untargeted attack since it is trivial for the proposed framework, which can easily achieve a 100% attack success rate, while Ebrahimi et al.(2017) could achieve 76.24% attack success rate for text summarization and 98.8% success rate for machine translation with 1 word change.”