Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual

By He He, Sheng Zha, Haohan Wang. 2019

217 citations recorded by the source corpus.

Statistical natural language inference (NLI) models are susceptible to learning dataset bias: superficial cues that happen to associate with the label on a particular dataset, but are not useful in general, e.g., negation words indicate contradiction. As exposed by several recent challenge datasets, these models perform poorly when such association is absent, e.g., predicting that “I love dogs.” contradicts “I don’t love cats.”. Our goal is to design learning algorithms that guard against known dataset bias. We formalize the concept of dataset bias under the framework of distribution shift and present a simple debiasing algorithm based on residual fitting, which we call DRiFt. We first learn a biased model that only uses features that are known to relate to dataset bias. Then, we train a debiased model that fits to the residual of the biased model, focusing on examples that cannot be predicted well by biased features only. We use DRiFt to train three high-performing NLI models on two benchmark datasets, SNLI and MNLI. Our debiased models achieve significant gains over baseline models on two challenge test sets, while maintaining reasonable performance on the original test sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “…word embeddings (Bolukbasi et al., 2016; Brunet et al., 2018; Zhao et al., 2019), visual semantic role labeling (Zhao et al., 2017), natural language inference (He et al., 2019), abusive language classiﬁcation (Park et al., 2018 Table 1: Counts of gendered words in several dialogue datasets.”
Adversarial Filters of Dataset Biases (2020, background): “Also related are approaches in He et al. (2019) where speciﬁc NLI biases are targeted; we show AFL ITE is capable of removing any spurious bias.”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “, methods that aim to decrease over-reliance on spurious correlations in NLU models (Clark et al., 2019; He et al., 2019; Karimi Mahabadi et al., 2020; Bras et al., 2020).”
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases (2019, background): “Concurrently, He et al. (2019) also suggested using a product-of-experts ensemble to train unbiased models, but we consider a wider variety of ensembling approaches and test on additional domains.”
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020, methodology): “We hypothesize that the baseline model exploits statistical pattern, or dataset bias presented in questions and that our method reduces the model’s tendency to exploit those spurious statistical patterns (He et al., 2019; Elkahky et al., 2018), which leads to large improvements in consistency.”
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020, background): “This issue has also been referred to as annotation artifacts (Guru-rangan et al., 2018), dataset bias (He et al., 2019; Clark et al., 2019), and group shift (Oren et al., 2019; Sagawa et al., 2020) in the literature.”
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020, methodology): “Even more recently, Clark et al. (2019) and He et al. (2019) propose to robustify the task model with the help of an additional simple model, using ensembling to encourage cooperation of the two models.”
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, background): “…include joint training (and discarding) of robust models together with models that are designed to exploit the dataset biases (Clark et al., 2019; He et al., 2019), re-weighting the loss function to incorporate the bias in the data (Schuster et al., 2019; Zhang et al., 2019b), parameter…”
Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback (2023, methodology): “Typically, this kind of method (Clark et al., 2019; He et al., 2019) usually contains two stages.”
Rethinking machine unlearning for large language models (2024, unknown): “Unlearning can be a natural solution for both types of attacks given the existing success of unlearning as a defense against adversarial attacks in other domains (Wang et al., 2019; Li et al., 2021; Liu et al., 2022b; Jia et al., 2023).”
Look at the First Sentence: Position Bias in Question Answering (2020, methodology): “Our proposed methods are based on the bias ensemble method (Clark et al., 2019; He et al., 2019).”
Supervising Model Attention with Human Explanations for Robust Natural Language Inference (2021, methodology): “…within NLI is by creating a weak model to intentionally learn a bias, then encouraging a target model to have low similarity to this weak model (He, Zha, and Wang 2019; Clark, Yatskar, and Zettlemoyer 2019; Ma-habadi, Belinkov, and Henderson 2020; Utama, Moosavi, and Gurevych 2020b; Sanh et…”
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures (2020, methodology): “We compare our results with the conﬁdence regularization approach of Utama et al. (2020a) and the product-of-expert approach (He et al., 2019; Clark et al., 2019).”
Analyzing Stereotypes in Generative Text Inference Tasks (2021, background): “There has simultaneously been a range of work aimed to mitigate problems of stereotyping in NLP systems, including many in the space of text generation (Sheng et al., 2020; He et al., 2019; Clark et al., 2019; Huang et al., 2020).”
Distilling Model Failures as Directions in Latent Space (2022, unknown): “Outside the fairness literature, de-biasing approaches in machine learning have been proposed for mitigating the effect of known biases during training (Sagawa et al., 2020; Yang et al., 2019; He et al., 2019; Belinkov et al., 2019; Clark et al., 2019).”
Natural Language Inference with a Human Touch: Using Human Explanations to Guide Model Attention (2021, background): “While research to date on creating more robust NLI models has mostly focused on preventing models learning from biases or artefacts in their datasets (Sanh et al., 2020; Belinkov et al., 2019a; Mahabadi et al., 2020; Utama et al., 2020; He et al., 2019), we take a different approach.”
Learning from others' mistakes: Avoiding dataset biases without modeling them (2020, methodology): “Both He et al. (2019) and Clark et al. (2019) protect against the reliance on this cheating feature by ensembling the main model with a biased model that only uses the hypothesis (or its ﬁrst token).”
Evaluating Gender Bias in Natural Language Inference (2021, background): “Typically, NLI systems are trained on datasets collected using large-scale crowd-sourcing techniques, which has its own fair share of issues resulting in the introduction of lexical bias in the trained models (He et al., 2019b; Clark et al., 2019).”
Syntactic Data Augmentation Increases Robustness to Inference Heuristics (2020, methodology): “…2019; Pang et al., 2019), auxiliary models designed to capture pre-defined shallow heuristics so that the main model can focus on robust strategies (Clark et al., 2019; He et al., 2019; Mahabadi and Henderson, 2019), and methods to up-weight difficult training examples (Yaghoobzadeh et al., 2019).”