Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

By Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits. 2019

1,382 citations recorded by the source corpus.

Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate three advantages of this framework: (1) effective—it outperforms previous attacks by success rate and perturbation rate, (2) utility-preserving—it preserves semantic content, grammaticality, and correct types classified by humans, and (3) efficient—it generates adversarial text with computational complexity linear to the text length.1

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Training for Large Neural Language Models (2020, background): “However, such models can still suffer catastrophic loss in adversarial scenarios (Nie et al., 2019; Hsieh et al., 2019; Madry et al., 2017; Jin et al., 2019), with attacks as simple as replacing a few words in input sentences while preserving the semantics.”
A Primer in BERTology: What We Know About How BERT Works (2020, background): “In particular, they were shown to rely on shallow heuristics in natural language inference (McCoy et al., 2019b; Zellers et al., 2019; Jin et al., 2020), reading comprehension (Si et al.”
When BERT Plays the Lottery, All Tickets Are Winning (2020, background): “on dataset biases (McCoy et al., 2019b; Rogers et al., 2020a; Jin et al., 2020; Niven and Kao, 2019; Zellers et al., 2019).”
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, background): “(III) TextFooler (Jin et al., 2020) is the state-of-the-art word-level adversarial attack method to generate adversarial examples.”
ReCode: Robustness Evaluation of Code Generation Models (2022, background): “For example, LLMs can be easily fooled by synonym replacement (Jin et al., 2020; Zang et al., 2020).”
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning (2020, methodology): “However, recent works have shown the ﬂaws of these PLMs in terms of robustness (Jin et al., 2020; Zang et al., 2020; Si et al., 2020; Li et al., 2020; Garg and Ramakrishnan, 2020; Tan et al., 2020; Wang et al., 2020a), where PLMs ﬁne-tuned on various downstream datasets are easily fooled by…”
Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs (2021, background): “Recently, a lot of work show that NLP models can be fooled by different types of adversarial attacks (Alzantot et al., 2018; Ebrahimi et al., 2018; Iyyer et al., 2018; Tan et al., 2020; Jin et al., 2020).”
BAE: BERT-based Adversarial Examples for Text Classification (2020, methodology): “Jin et al. (2019) combine both these works by proposing TextFooler, a strong black-box attack baseline for text classiﬁcation models.”
A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement (2022, methodology): “Such vulnerability has been exposed in many NLP tasks including text classiﬁcation [ Jin et al. , 2020 ] , machine translation [ Zhang et al. , 2021 ] , dependency parsing [ Zheng et al. , 2020 ] , reading comprehension [ Lin et al. , 2021 ] and dialogue systems [ Cheng et al. , 2019 ] .”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, methodology): “Adversarial methods are often used to find weaknesses in machine learning models (Jin et al., 2020; Garg and Ramakrishnan, 2020).”
Gradient-based Adversarial Attacks against Text Transformers (2021, methodology): “We compare against several recent attacks on text transformers: TextFooler (Jin et al., 2020), BAE (Garg and Ramakrishnan, 2020), and BERT-Attack (Li et al., 2020b).”
"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models (2023, neutral): “Besides jailbreak attacks, language models also face other attacks, such as prompt injection [34, 55], backdoor [21, 27], data extraction [26, 44], obfuscation [39], membership inference [50, 66], and adversarial attacks [25, 38, 71].”
Text Generation with Efficient (Soft) Q-Learning (2021, result): “This is a more challenging setting compared to previous instance-speciﬁc attack [35, 24, 12] where Figure 4: Entailment attack performance against diversity (average of H 1 and H 2 ).”
On Evaluating Adversarial Robustness of Large Vision-Language Models (2023, background): “Nevertheless, Alzantot et al. [4] first construct adversarial examples on sentiment analysis and textual entailment tasks, while Jin et al. [36] report that BERT can be evaded through natural language attacks.”
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets (2020, background): “Adversarial methods generate test instances automatically, which can be applied to augment the training data (Jin et al., 2019a; Dinan et al., 2019).”