Explaining The Efficacy of Counterfactually-Augmented Data

By Divyansh Kaushik, Amrith Rajagopal Setlur, Eduard H. Hovy, Zachary Chase Lipton. 2020

91 citations recorded by the source corpus.

In attempts to produce machine learning models less reliant on spurious patterns in training data, researchers have recently proposed a human-in-the-loop process for generating counterfactually augmented datasets. As applied in NLP, given some documents and their (initial) labels, humans are tasked with revising the text to make a (given) counterfactual label applicable. Importantly, the instructions prohibit edits that are not necessary to flip the applicable label. Models trained on the augmented (original and revised) data have been shown to rely less on semantically irrelevant words and to generalize better out-of-domain. While this work draws on causal thinking, casting edits as interventions and relying on human understanding to assess outcomes, the underlying causal model is not clear nor are the principles underlying the observed improvements in out-of-domain evaluation. In this paper, we explore a toy analog, using linear Gaussian models. Our analysis reveals interesting relationships between causal models, measurement noise, out-of-domain generalization, and reliance on spurious signals. Interestingly our analysis suggests that data corrupted by adding noise to causal features will degrade out-of-domain performance, while noise added to non-causal features may make models more robust out-of-domain. This analysis yields interesting insights that help to explain the efficacy of counterfactually augmented data. Finally, we present a large-scale empirical study that supports this hypothesis.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

A Survey of Data Augmentation Approaches for NLP (2021, background): “Models can also learn to combine together simpler DA primitives (Cubuk et al., 2018; Ratner et al., 2017) or add human-in-the-loop (Kaushik et al., 2020, 2021).”
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study (2021, methodology): “Kaushik et al. (2020, 2021) collect counterfactually augmented data (CAD) by asking crowdworkers to edit existing documents to make counterfactual labels applicable, showing that models trained on CAD generalize better out-of-domain.”
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks (2023, background): “Another body of work examines the robustness of model predictions using counterfactual data (Kaushik et al., 2020, 2021; Gardner et al., 2020).”
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2021, background): “Similarly, sensitivity tests can be viewed as evaluations of counterfactuals X(Y = ỹ), in which the label Y is changed but all other causal influences on X are held constant (Kaushik et al., 2020).”
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022, background): “…performance metrics (and associated robust optimizers) can be seen as enforcing or testing task-speciﬁc invariance properties that provide robustness against known distributional shifts (e.g., Lu et al., 2020; Ribeiro et al., 2020; Kaushik et al., 2021; Koh et al., 2021; Veitch et al., 2021).”