Semantically Equivalent Adversarial Rules for Debugging NLP models

By Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin. 2018

557 citations recorded by the source corpus.

Complex machine learning models for NLP are often brittle, making different predictions for input instances that are extremely similar semantically. To automatically detect this behavior for individual instances, we present semantically equivalent adversaries (SEAs) – semantic-preserving perturbations that induce changes in the model’s predictions. We generalize these adversaries into semantically equivalent adversarial rules (SEARs) – simple, universal replacement rules that induce adversaries on many instances. We demonstrate the usefulness and flexibility of SEAs and SEARs by detecting bugs in black-box state-of-the-art models for three domains: machine comprehension, visual question-answering, and sentiment analysis. Via user studies, we demonstrate that we generate high-quality local adversaries for more instances than humans, and that SEARs induce four times as many mistakes as the bugs discovered by human experts. SEARs are also actionable: retraining models using data augmentation significantly reduces bugs, while maintaining accuracy.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Pretrained Transformers Improve Out-of-Distribution Robustness (2020, background): “Adversarial examples can be created for NLP models by inserting phrases (Jia and Liang, 2017; Wallace et al., 2019), paraphrasing questions (Ribeiro et al., 2018), and reducing inputs (Feng et al., 2018).”
Finding Generalizable Evidence by Learning to Convince Q&A Models (2019, background): “Neural networks are susceptible to adversarial examples—changes to the input which do or do not change the network’s prediction in surprising ways (Szegedy et al., 2014; Jia and Liang, 2017; Ribeiro et al., 2018; Alzantot et al., 2018).”
Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, background): “[2, 16] studied how to craft adversarial examples for image captioning, and [10] investigated how to derive adversarial rules to attack VQA systems.”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, background): “On the other hand, recurring ﬁndings reveal that such models are brittle when provided with out-of-domain [1, 2, 3] or adversarial examples [4, 5, 6, 7, 8, 9, 10], and make silly mistakes that contradict basic notions of human common sense.”
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021, background): “For example, adversarial generators aim to maintain semantics while changing model predictions (Ribeiro et al., 2018; Iyyer et al., 2018; Li et al., 2021), whereas concurrent work to our own (Madaan et al.”
Measuring and Improving Consistency in Pretrained Language Models (2021, methodology): “Ribeiro et al. (2018) proposed another method for creating paraphrases and performed a similar analysis for visual question answering and sentiment analysis.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “, 2018) Black 7 Word Text classification, sentiment (Ribeiro et al., 2018) Black 7 Word Reading comprehension, visual QA, sentiment (Niu and Bansal, 2018) Black 7 Word Dialogue (Chen et al.”
Universal Adversarial Triggers for Attacking and Analyzing NLP (2019, methodology): “The construction of NLP datasets can lead to dataset biases or “artifacts”.”
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data (2019, background): “For example, otherwise state-of-the-art models have been shown to be vulnerable to synthetic transformations such as distractor phrases [11, 27], to misclassify paraphrased task [10, 22] and to fail on template-based modifications [25].”
BLEURT: Learning Robust Metrics for Text Generation (2020, methodology): “Generating synthetic data via para-phrases and perturbations has been commonly used for generating adversarial examples (Jia and Liang, 2017; Iyyer et al., 2018; Belinkov and Bisk, 2018; Ribeiro et al., 2018), an orthogonal line of research.”
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (2020, background): “, 2019) or adversarial changes (Ribeiro et al., 2018; Iyyer et al., 2018), fairness (Prabhakaran et al.”
Counterfactual Fairness in Text Classification through Robustness (2018, methodology): “One could also use criteria for selecting semantically equivalent adversarial examples as in [20], to evaluate whether counterfactual examples are logical.”
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020, methodology): “However, note that this problem is not unique to contrast sets—similar issues hold for the original test set as well as adversarial test sets (Jia and Liang, 2017), challenge sets (Naik et al., 2018), and input perturbations (Ribeiro et al., 2018a; Feng et al., 2018).”
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly (2019, background): “Ribeiro et al. (2018) test for comprehension of minimally modified sentences in an adversarial setup while trying to keep the overall semantics the same.”
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets (2019, background): “These challenges have been used as evidence that current systems are brittle (Belinkov and Bisk, 2018; Mudrakarta et al., 2018; Zhao et al., 2018; Glockner et al., 2018; Ebrahimi et al., 2018; Ribeiro et al., 2018, 1 Often referred to as “adversarial datasets” or “attacks”.”
PAWS: Paraphrase Adversaries from Word Scrambling (2019, background): “Example perturbation rules include adding noise to inputs (Jia and Liang, 2017; Chen et al., 2018), word replacements (Alzantot et al., 2018; Ribeiro et al., 2018), and syntactic transformation (Iyyer et al., 2018).”
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “Synthetic Data Generative Adversarial Networks [57], [58], [59], [60], [61], [62] Policies [63], [64] Image [65], [66], [67], [68], [69], [70], [71] Text [72], [73], [74]”
Deep Weighted Averaging Classifiers (2018, methodology): “…models, in a way that provides some utility to the user or developer of a model beyond what is attainable through more direct means [2, 27, 30, 42–44 In this paper, we propose a method which, like those in the former category, offers an explanation that is transparent (in that the complete…”