Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

By Divyansh Kaushik, Eduard H. Hovy, Zachary Chase Lipton. 2019

662 citations recorded by the source corpus.

Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are due to confounding (e.g., a common cause), but not direct or indirect causal effects. In this paper, we focus on natural language processing, introducing methods and resources for training models less sensitive to spurious patterns. Given documents and their initial labels, we task humans with revising each document so that it (i) accords with a counterfactual target label; (ii) retains internal coherence; and (iii) avoids unnecessary changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are less sensitive to this signal. Both datasets are publicly available.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Experience Grounds Language (2020, background): “Perhaps this is to be expected, since they are tested in terms of Independent and Identi-cally Distributed held-out sets, rather than queried on points meant to probe the granularity of the distinctions they make (Kaushik et al., 2020; Gardner et al., 2020).”
Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “Kaushik et al. (2019) offer a causal account of spurious patterns, and counterfac-tually augment NLI datasets by editing examples to break the model.”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “Most relevantly, Kaushik et al. (2020) and Gardner et al. (2020) propose augmenting NLP datasets with contrastive cases for training more generalisable models and enabling more meaningful evaluation.”
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor (2022, methodology): “A common approach is to automatically augment existing datasets (Anaby-Tavor et al., 2020; Andreas, 2020; Yang et al., 2020; Kaushik et al., 2020; Lee et al., 2021, inter alia).”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “, 2019), and new systems for developing customized stress tests have been proposed (Ribeiro et al., 2020; Kaushik et al., 2020).”
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, neutral): “, “contrastive” [22] or “counterfactual” [38] examples).”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, methodology): “On the other hand, recurring findings reveal 18 that such models are brittle when provided with out-of-domain [1, 2, 3] or adversarial examples 19 [4, 5, 6, 7, 8, 9, 10], and make silly mistakes that contradict basic notions of human common sense.”
Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “Making such ﬁne-grained perturbations allows for more careful evaluation and improvement of models’ language understanding (Kaushik et al., 2020; Wu et al., 2021).”
A Survey of Data Augmentation Approaches for NLP (2021, background): “Models can also learn to combine together simpler DA primitives (Cubuk et al., 2018; Ratner et al., 2017) or add human-in-the-loop (Kaushik et al., 2020, 2021).”
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021, methodology): “(Kaushik et al., 2020) quantifier A dog is ) Three dogs are embraced by the woman.”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, background): “The perturbations are challenging ‘contrast sets’, which manipulate the original text just enough to flip the label (e.g. from ‘Hate’ to ‘Not Hate’) (Kaushik et al., 2019; Gardner et al., 2020).”
Fast Model Editing at Scale (2021, background): “Counterfactual data augmentation (Kaushik et al., 2020) may be useful for constructing richer evaluation cases for edit generality.”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, background): “Kaushik et al., 2020; Vig et al., 2020) or in specific neurons (Vig et al., 2020), our intervention is done on the representation layers.”
CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, methodology): “Specifically, researchers have been looking into methods of measuring other confounders via text (Pennebaker, Francis, and Booth 2001; Saha et al. 2019), or using text as confounders (Johansson, Shalit, and Sontag 2016; De Choudhury et al. 2016; Roberts, Stewart, and Nielsen 2018).”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “A common strategy to mitigate biases is to change the training data (e.g., Lu et al., 2018; Hall Maudslay et al., 2019; Zhao et al., 2018a; Kaushik et al., 2019), the training process (e.g., Huang et al., 2019; Qian et al., 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al.,…”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Recent work has also focused on evaluating model diagnostics through counterfactual augmentation (Kaushik et al., 2020), decision boundary analysis (Gardner et al., 2020; Swayamdipta et al., 2020), and behavioural testing (Ribeiro et al., 2020).”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, methodology): “Recent papers also outline methodologies for leveraging human insight in the manual construction of counterfactual examples that complement instances in natural language processing datasets to promote better generalization [Gardner et al., 2020, Kaushik et al., 2020].”