Annotation Artifacts in Natural Language Inference Data

By Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, Noah A. Smith. 2018

1,309 citations recorded by the source corpus.

Large-scale datasets for natural language inference are created by presenting crowd workers with a sentence (premise), and asking them to generate three new sentences (hypotheses) that it entails, contradicts, or is logically neutral with respect to. We show that, in a significant portion of such data, this protocol leaves clues that make it possible to identify the label by looking only at the hypothesis, without observing the premise. Specifically, we show that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI (Bowman et. al, 2015) and 53% of MultiNLI (Williams et. al, 2017). Our analysis reveals that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes. Our findings suggest that the success of natural language inference models to date has been overestimated, and that the task remains a hard open problem.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, methodology): “Whereas fastText, a bag of n -gram model, obtains 67.0% accuracy on SNLI versus a 34.3% baseline (Gururangan et al., 2018), fastText obtains only 29.0% accuracy on Swag .”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, methodology): “Auditing In light of recent work showing that crowdsourced data often contains artifacts which can be exploited to perform well without solving the intended task (Schwartz et al., 2017; Gururangan et al., 2018), we perform an audit of our manually curated data as a sanity check.”
Language Models are Few-Shot Learners (2020, background): “The main disadvantages are the need for a new large dataset for every task, the potential for poor generalization out-of-distribution [MPL19], and the potential to exploit spurious features of the training data [GSL + 18, NK19].”
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, background): “It would be surprising if low-bias learners such as neural networks did not notice such a correlation, and indeed they do, leading them to respond CONTRADICTION with high probability any time the hypothesis contains a negation word (Gururangan et al., 2018; Poliak et al., 2018).”
Experience Grounds Language (2020, methodology): “This disconnect is driven home by studies of bias (Gururangan et al., 2018; Glockner et al., 2018) and techniques like Adversarial Filtering (Zellers et al., 2018, 2019b), which elucidate the biases models exploit in lieu of understanding.”
Pretrained Transformers Improve Out-of-Distribution Robustness (2020, background): “Annota-tors can accidentally leave unintended shortcuts in datasets that allow models to achieve high accuracy by effectively “cheating” (Cai et al., 2017; Gururangan et al., 2018; Min et al., 2019).”
PIQA: Reasoning about Physical Commonsense in Natural Language (2019, methodology): “As noted previously, we use AFLite (Sakaguchi et al. 2020) to remove stylistic artifacts and trivial examples from the data, which have been shown to artiﬁcially inﬂate model performance on previous NLI benchmarks (Poliak et al. 2018; Gururangan et al. 2018).”
Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, methodology): “Following previous observations that models might exploit spurious biases in NLI hypotheses, (Gururangan et al., 2018; Poliak et al., 2018), we”
WinoGrande (2019, methodology): “Several recent studies (Gururangan et al. 2018; Poliak et al. 2018; Tsuchiya 2018; Niven and Kao 2019; Geva, Goldberg, and Berant 2019) have reported the presence of annotation artifacts in large-scale datasets.”
Probing Neural Network Comprehension of Natural Language Arguments (2019, background): “There is much recent work focusing on statistical cues in datasets in vision (Jo and Bengio, 2017) and NLP (Sanchez et al., 2018; McCoy et al., 2019; Gururangan et al., 2018; Glockner et al., 2018; Poliak et al., 2018; Rajpurkar et al., 2018; Jia and Liang, 2017).”
HellaSwag: Can a Machine Really Finish Your Sentence? (2019, background): “5These biases are similar to those in NLI datasets, as found by Gururangan et al. (2018); Poliak et al.”
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (2019, methodology): “The success of MultiNLI might also be surprising given recent concerns about the generalization abilities of models trained on it (Glockner et al., 2018), particularly related to “annotation artifacts” caused by using crowd workers to write the hypothesis statements (Gururangan et al., 2018).”
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019, background): “A recent line of work (Gururangan et al., 2018; Kaushik and Lipton, 2018) has identiﬁed that popular synthetic NLP datasets (such as SQuAD (Ra-jpurkar et al., 2016) or SNLI (Bowman et al., 2015)) are prone to have artifacts and annotation biases which can be exploited by supervised algorithms that…”
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, background): “In the NLI literature, Poliak et al. (2018b) and Gururangan et al. (2018) show that, due to biases in NLI datasets, it is possible to achieve reasonable success at NLI by only looking at the hypothesis.”
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering (2018, background): “3 Motivated by recent ﬁndings of gameability of NLP datasets (Gururangan et al., 2018), we also develop and evaluate simple, attention-based, neural baselines including a plausible answer detector (which ignores the question text completely) and an odd-one-out solver .”
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge (2018, background): “In-deed, it is easy to mistake progress on these datasets as implying equal progress on easy and hard questions, while in reality progress may be heavily concentrated on easy questions alone (Gururangan et al., 2018), leaving more difﬁcult challenges unaddressed.”
XNLI: Evaluating Cross-lingual Sentence Representations (2018, background): “The top hypothesis words indicative of the class label – scored using the mutual information between each word and class in the corpus – are similar across languages, and overlap those of the MultiNLI corpus (Gururangan et al., 2018).”
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, background): “Nevertheless, there has been mounting evidence in recent work suggesting that the performance of these models as measured by the benchmark datasets may be overestimated, non-generalizable and at least partially driven by exploiting existing spurious correlations in training datasets [21, 22, 40, 47,”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “” Building benchmarks with a very specific subset of American English text in natural language processing, and the built-in bias of annotation artifacts [Gururangan et al., 2018] results in inherently subjective outcomes [Waseem et al.”
Are NLP Models really able to Solve Simple Math Word Problems? (2021, background): “In Gururangan et al. (2018), the model is Following Gururangan et al. (2018), we attempt to understand the extent to which SOTA models rely on the presence of simple heuristics in the body to predict correctly.”