Learning from others' mistakes: Avoiding dataset biases without modeling them

By Victor Sanh, Thomas Wolf, Yonatan Belinkov, Alexander M. Rush. 2020

124 citations recorded by the source corpus.

State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying task. Previous work has demonstrated effective methods to circumvent these issues when knowledge of the bias is available. We consider cases where the bias issues may not be explicitly identified, and show a method for training models that learn to ignore these problematic correlations. Our approach relies on the observation that models with limited capacity primarily learn to exploit biases in the dataset. We can leverage the errors of such limited capacity models to train a more robust model in a product of experts, thus bypassing the need to hand-craft a biased model. We show the effectiveness of this method to retain improvements in out-of-distribution settings even if no particular bias is targeted by the biased model.

Subfields: Artificial Intelligence, General Social Sciences, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Evaluation & Benchmarks

Citations in the corpus

Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets (2021, background): “Previously proposed debiasing methods include [6]’s foundational work to debias word embeddings; [29]’s use of product of experts to train a model to avoid dataset biases; [39]’s human-and-model-in-the-loop technique to better train and evaluate models without toxic and unwanted behavior; [23]’s use”
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets (2022, methodology): “Furthermore, we compare our method to strong debiasing strategies from the literature (Belinkov et al., 2019b; Stacey et al., 2020; Clark et al., 2019; Karimi Mahabadi et al., 2020; Utama et al., 2020; Sanh et al., 2021; Ghaddar et al., 2021).”
Distilling Model Failures as Directions in Latent Space (2022, unknown): “Other works directly train a second model which “emphasizes” the examples on which a base model struggles (Nam et al., 2020; Hashimoto et al., 2018; Bao & Barzilay, 2022; Kim et al., 2019; Utama et al., 2020; Sanh et al., 2020).”
Supervising Model Attention with Human Explanations for Robust Natural Language Inference (2021, methodology): “The in-distribution beneﬁts from training with the explanations contrast with previous work on model robustness, with most work involving a trade-off between robustness and in-distribution performance (Sanh et al. 2020; Mahabadi, Belinkov, and Henderson 2020; Belinkov et al. 2019a).”
Natural Language Inference with a Human Touch: Using Human Explanations to Guide Model Attention (2021, methodology): “This is in contrast to most prior work improving NLI robustness (Belinkov et al., 2019b; Mahabadi et al., 2020; Sanh et al., 2020).”