Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases

By Christopher Clark, Mark Yatskar, Luke S. Zettlemoyer. 2019

525 citations recorded by the source corpus.

State-of-the-art models often make use of superficial patterns in the data that do not generalize well to out-of-domain or adversarial settings. For example, textual entailment models often learn that particular key words imply entailment, irrespective of context, and visual question answering models learn to predict prototypical answers, without considering evidence in the image. In this paper, we show that if we have prior knowledge of such biases, we can train a model to be more robust to domain shift. Our method has two stages: we (1) train a naive model that makes predictions exclusively based on dataset biases, and (2) train a robust model as part of an ensemble with the naive one in order to encourage it to focus on other patterns in the data that are more likely to generalize. Experiments on five datasets with out-of-domain test sets show significantly improved robustness in all settings, including a 12 point gain on a changing priors visual question answering dataset and a 9 point gain on an adversarial question answering test set.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “Dialogue research has found that, while incorporating personas increases engagingness and improves consistency (Zhang et al., 2018; Shuster et al., 2018; Mazar´e et al., 2018; Olabiyi et al., 2018; Li et al., 2016b), they can also crystallize gender bias (Clark et al., 2019; Henderson et al., 2018).”
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts (2021, background): “Another supporting evidence is the issue of strong language priors (well-known in the VQA community), which led to interest in adversarial test sets and other methods to overcome strong language biases [1, 30, 14, 7].”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “tures and use it in ensemble learning (Clark et al., 2019).”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “, methods that aim to decrease over-reliance on spurious correlations in NLU models (Clark et al., 2019; He et al., 2019; Karimi Mahabadi et al., 2020; Bras et al., 2020).”
HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data (2020, background): “Annotation De-biasing As has been suggested in previous papers (Kaushik and Lipton, 2018; Chen and Durrett, 2019; Clark et al., 2019), the existing benchmarks on multi-hop reasoning question answering have annotation biases, which makes designing multi-hop models unnecessary.”
Competency Problems: On Finding and Removing Artifacts in Language Data (2021, methodology): “…edits for reducing single feature artifacts using locally edited samples generated from two datasets: (1) the Boolean Questions dataset (BoolQ; Clark et al., 2019a), which consists of pairs of paragraphs ( p ) and questions ( q ), where each q has a binary answer that can be found by…”
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020, background): “, 2018), dataset bias (He et al., 2019; Clark et al., 2019), and group shift (Oren et al.”
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision (2020, methodology): “Several models and training meth-ods [11,15,16,26,27,40,52] have been proposed with signiﬁcant improvements.”
Handling Bias in Toxic Speech Detection: A Survey (2022, methodology): “Meanwhile, the LearnedMixIn method [17] that adopts model regularisation provides the smallest percentage of black tweets samples wrongly labelled as toxic.”
Fool Me Twice: Entailment from Wikipedia Gamification (2021, methodology): “Like the techniques proposed by Clark et al. (2019), the claim-only classiﬁer can also be used on both F OOL M E T WICE and FEVER to split the dev sets into “easy” and “hard” partitions: The EASY partition contains all examples correctly clas-siﬁed by a claim-only classiﬁer, and the HARD partition…”
Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making (2021, background): “In other words, the test set may differ from the training set, so the patterns that AI identifies can fail during testing, leading to a substantial drop in AI performance [10, 25, 46].”
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers (2022, background): “This often makes visual question answering models depend on statistical bias instead of desired reasoning process [1,14,15,23].”
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures (2020, methodology): “We compare our results with the conﬁdence regularization approach of Utama et al. (2020a) and the product-of-expert approach (He et al., 2019; Clark et al., 2019).”
On the Paradox of Learning to Reason from Data (2022, background): “…identiﬁed in typical NLP datasets, which are often due to biases in the dataset collection/annotation process (Gururangan et al., 2018; Clark et al., 2019; He et al., 2019), statistical features inherently exist in reasoning problems and are not speciﬁc to certain data distributions.”
Supervising Model Attention with Human Explanations for Robust Natural Language Inference (2021, background): “…a weak model to intentionally learn a bias, then encouraging a target model to have low similarity to this weak model (He, Zha, and Wang 2019; Clark, Yatskar, and Zettlemoyer 2019; Ma-habadi, Belinkov, and Henderson 2020; Utama, Moosavi, and Gurevych 2020b; Sanh et al. 2020; Liu et al. 2020;…”