Adversarial Filters of Dataset Biases

By Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, Yejin Choi. 2020

248 citations recorded by the source corpus.

Large neural models have demonstrated human-level performance on language and vision benchmarks, while their performance degrades considerably on adversarial or out-of-distribution samples. This raises the question of whether these models have learned to solve a dataset rather than the underlying task by overfitting to spurious dataset biases. We investigate one recently proposed approach, AFLite, which adversarially filters such dataset biases, as a means to mitigate the prevalent overestimation of machine performance. We provide a theoretical understanding for AFLite, by situating it in the generalized framework for optimum bias reduction. We present extensive supporting evidence that AFLite is broadly applicable for reduction of measurable dataset biases, and that models trained on the filtered datasets yield better generalization to out-of-distribution tasks. Finally, filtering results in a large drop in model performance (e.g., from 92% to 62% for SNLI), while human performance still remains high. Our work thus shows that such filtered datasets can pose new research challenges for robust generalization by serving as upgraded benchmarks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Understanding Dataset Difficulty with V-Usable Information (2021, neutral): “We additionally consider slices in the dataset based on dataset cartography (Swayamdipta et al., 2020), which uses training dynamics to differentiate instances via their (1) confidence (i.e., mean probability of the correct label across epochs), and (2) variability (i.e., variance of the former).”
Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, methodology): “[8] propose AFLite, an adversarial filtering algorithm for both refining training sets and creating harder test sets.”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, methodology): “To generate more challenging datasets, recent research proposed algorithmic methods for bias reduction, such as adversarial filtering [11, 12, 13, 14, 15].”
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark (2021, background): “To address this issue, recent commonsense benchmarks often use adversarial ﬁltering (Zellers et al. 2018; Le Bras et al. 2020): a family of techniques that remove easily predicted examples from datasets.”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, methodology): “(Swayamdipta et al., 2020) of an existing dataset relative to a trained model, (1) we automatically identify pockets of data instances exemplifying challenging reasoning patterns.”
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2021, methodology): “While manual dataset creation is expensive and complex (Schwartz et al., 2017; Agrawal et al., 2018; Tsuchiya, 2018; Bras et al., 2020), crowdsourcing is the most popular method for goal-oriented, high quality/coverage datasets.”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, methodology): “This algorithm is applied by Le Bras et al. [2020] to a variety of natural language processing datasets, and they ﬁnd that training models on adversarially ﬁltered data leads to better generalization to out-of-distribution data.”
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021, background): “1 The DynaBench approach falls into the broader category of adversarial ﬁltering (Paperno et al., 2016; Zellers et al., 2018; Nie et al., 2020; Le Bras et al., 2020).”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “, methods that aim to decrease over-reliance on spurious correlations in NLU models (Clark et al., 2019; He et al., 2019; Karimi Mahabadi et al., 2020; Bras et al., 2020).”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, background): “In adversarial517 filtering (Le Bras et al., 2020; Zellers et al., 2018),518 one takes an existing dataset and trains a model on519 the most difficult subportion of the data.”
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020, background): “, 2018), AFLite (LeBras et al., 2020), AL-uncertainty (Joshi et al.”
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021, background): “, 2020a), or adversarially removing troublesome training examples (Bras et al., 2020) or manually collecting new ones (Nie et al.”
G-DAug: Generative Data Augmentation for Commonsense Reasoning (2020, background): “We hypothesize that filtering out detrimental synthetic training examples can boost downstream performance (Bras et al., 2020).”
CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge (2021, methodology): “We do not take a model-in-the-loop approach [Zellers et al., 2018, 2019, Nie et al., 2020, Bras et al., 2020] during data collection in order to keep our dataset organic, meaning that sentences preserve the original data distribution given by annotators.”
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2020, methodology): “usage of unsupervised methods (Zhao and Chang, 2020), or on repurposing of unsupervised methods for generalisation and dataset artifacts (Bras et al., 2020) would also be valuable, though those also would benefit from subgroup identity annotation to make their results more interpretable.”
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets (2020, methodology): “Bras et al. (2020) uses an adversarial ﬁltering algorithm to reduce spurious bias in the dataset to avoid models relying on such patterns.”