BAE: BERT-based Adversarial Examples for Text Classification

By Siddhant Garg, Goutham Ramakrishnan. Conference on Empirical Methods in Natural Language Processing, 2020

669 citations recorded by the source corpus.

Modern text classification models are susceptible to adversarial examples, perturbed versions of the original text indiscernible by humans but which get misclassified by the model. We present BAE, a powerful black box attack for generating grammatically correct and semantically coherent adversarial examples. BAE replaces and inserts tokens in the original text by masking a portion of the text and leveraging a language model to generate alternatives for the masked tokens. Compared to prior work, we show that BAE performs a stronger attack on three widely used models for seven text classification datasets.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/06a427e1688f92053a38c73cb4e0da25177c89e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021, positive): “However, recent works show that PLMs suffer from poor robustness when encountering adversarial examples (Jin et al., 2020; Li et al., 2020; Garg and Ramakrishnan, 2020; Zang et al., 2020; Lin et al., 2020a).”
Can Large Language Models Be an Alternative to Human Evaluations? (2023, background): “, 2019), and BAE (Garg and Ramakrishnan, 2020); these attacks are predominantly used as strong baselines in the literature of SSAs nowadays.”
Gradient-based Adversarial Attacks against Text Transformers (2021, methodology): “These constraints tend to produce out-of-context and unnatural changes that alter the semantic meaning of the perturbed text (Garg and Ramakrishnan, 2020).”
Explaining NLP Models via Minimal Contrastive Editing (MiCE) (2020, background): “…approaches to generating adversarial examples, which can exploit semantics-preserving operations (Ribeiro et al., 2018) such as paraphrasing (Iyyer et al., 2018) or word replacement (Alzantot et al., 2018; Ren et al., 2019; Garg and Ramakrishnan, 2020), cannot be used to generate contrastive edits.”
Explainability for Large Language Models: A Survey (2023, background): “By focusing on contextualized representations, a range of work adopting the mask-then-infill procedure has achieved state-of-the-art performance [Garg and Ramakrishnan 2020; Li et al. 2021a].”
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP (2020, methodology): “(Garg and Ramakrishnan, 2020) Untargeted Classification USE sentence encoding cosine similarity BERT Masked Token Prediction Greedy-WIR”
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning (2020, background): “However, recent works have shown the ﬂaws of these PLMs in terms of robustness (Jin et al., 2020; Zang et al., 2020; Si et al., 2020; Li et al., 2020; Garg and Ramakrishnan, 2020; Tan et al., 2020; Wang et al., 2020a), where PLMs ﬁne-tuned on various downstream datasets are easily fooled by…”
On Robust Prefix-Tuning for Text Classification (2022, background): “2019; Garg & Ramakrishnan, 2020), sentence-level modiﬁcation (Iyyer et al., 2018; Ribeiro et al., 2018; Xu et al., 2021), or a mixture of them (Ebrahimi et al., 2018; Li et al., 2019).”
Contextualized Perturbation for Textual Adversarial Attack (2020, background): “(2020) and Garg and Ramakrishnan (2020) are published concurrently to an initial report of this work.”
A Survey on Text Classification: From Traditional to Deep Learning (2020, neutral): “2019 RoBERTa [140] arXiv SA, QA [141] F1, Accuracy SQuAD, MNLI-m, SST-2 Trans 2020 GAN-BERT [142] ACL SA, NLI [143] F1, Accuracy SST-5, MNLI 2020 BAE [144] EMNLP SA, QA [145] Accuracy Amazon, Yelp, MR, MPQA 2020 ALBERT [146] ICLR SA, QA [147] F1, Accuracy SST, MNLI, SQuAD 2020 TG-Transformer [148] E”
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces (2020, background): “Therefore, such a model cannot be directly applied to generate multiple candidates as the substitution list (Li et al., 2020; Garg and Ramakrishnan, 2020) in adversarial attacks.”
Robustness Gym: Unifying the NLP Evaluation Landscape (2021, background): “∗ marked are CachedOperation s and the rest are SliceBuilder s. Transformations (e.g., S YNONYM A UG , K EYBOARD A UG ) [Ma, 2019], Attacks (T EXT A TTACK ) [Garg and Ramakrishnan, 2020, Morris et al., 2020], and Evaluation Sets [Bowman et al., 2015].”