Inherent Disagreements in Human Textual Inferences

By Ellie Pavlick, Tom Kwiatkowski. 2019

360 citations recorded by the source corpus.

Abstract We analyze human’s disagreements about the validity of natural language inferences. We show that, very often, disagreements are not dismissible as annotation “noise”, but rather persist as we collect more ratings and as we vary the amount of context provided to raters. We further show that the type of uncertainty captured by current state-of-the-art models for natural language inference is not reflective of the type of uncertainty present in human disagreements. We discuss implications of our results in relation to the recognizing textual entailment (RTE)/natural language inference (NLI) task. We argue for a refined evaluation objective that requires models to explicitly capture the full distribution of plausible human judgments.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “What are the impacts of annotation guidelines, annotator assumptions and perceptions (Olteanu et al., 2019; Sap et al., 2019; Geiger et al., 2020), and annotation aggregation processes (Pavlick and Kwiatkowski, 2019)?”
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, background): “There are also some examples in the dataset where the disagreement on the label for bias type should not be considered noise (Pavlick and Kwiatkowski, 2019).”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, neutral): “Upon inspecting the examples on which annotators disagreed, we observe that many examples represent genuinely ambiguous cases rather than careless mislabels, echoing previous findings (Pavlick and Kwiatkowski, 2019).”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “Pavlick and Kwiatkowski [2019] ﬁnd that human judgment on nat-ural language inference tasks is variable, and that machine evaluation on this task should reﬂect this variability.”
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021, background): “This can involve asking models to predict the empirical distribution of labels that trustworthy annotators assign (Pavlick and Kwiatkowski, 2019; Poesio et al., 2019), or allowing models to predict any of several answer choices that are supported by trustworthy annotators (as in the SQuAD benchmark)”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, background): “We focus on binary NLI—definitely entailing or not entailing—to minimize labeling disagreements stemming from the distinction between neutral and contradiction in three-way NLI (Pavlick and Kwiatkowski, 2019; Nie et al., 2020b).”
Probing Natural Language Inference Models through Semantic Fragments (2019, background): “Benchmark NLI annotations and judgements are often imperfect and error-prone (cf. Kalouli, de Paiva, and Real (2017), Pavlick and Kwiatkowski (2019)), partly due to the loose way in which the task is traditionally deﬁned (Dagan, Glickman, and Magnini 2005).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, background): “Such approaches ignore the variance in annotations (Pavlick and Kwiatkowski, 2019; Geva et al., 2019; Arhin et al., 2021; Akhtar et al., 2021) based on who the annotators are, and what their beliefs are.”
DynaSent: A Dynamic Benchmark for Sentiment Analysis (2020, background): “Distributional Labels We can repeat each example with each of its labels (de Marneffe et al., 2012; Pavlick and Kwiatkowski, 2019).”
Targeting the Benchmark: On Methodology in Current Natural Language Processing Research (2020, background): “5 For a language benchmark task, the argument roughly goes as follows (even if typically only 3 Pavlick and Kwiatkowski (2019), for example, show that the task of annotation textual entailments can lead to faultless annotator disagreements.”
Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application (2020, neutral): “There is a growing body of related literature showing that a single "true" labeled response is often unrealistic (Aroyo et al., 2019; Palomaki et al., 2018; Geva et al., 2019; Pavlick and Kwiatkowski, 2019).”
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, background): “Method used Used by / Investigated by HotPotQA Partial Baselines (Min et al., 2019; Sugawara et al., 2020; Chen and Durrett, 2019b) Adversarial Evaluation (Jiang and Bansal, 2019) Data Improvements (Jiang and Bansal, 2019) Arch/Training Improvements (Jiang and Bansal, 2019) Manual Analyses (Schlegel”
Evidence-based Verification for Real World Information Needs (2021, background): “Quality Controls Annotation of natural language inference depends on human judgement and is subject to disagreement (Pavlick and Kwiatkowski, 2019; Nie et al., 2020).”
ANLIzing the Adversarial Natural Language Inference Dataset (2020, neutral): “lyzing inherent disagreements between human annotators (Pavlick and Kwiatkowski, 2019).”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “However, this assumption has been 325 shown to be inaccurate (among others) for relation extraction [7], natural language inference [93, 85], 326 and complement coercion [34], and the extent to which it is true likely varies by task, instance, and 327 annotator.”
Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? (2020, result): “These accuracies are similar to those obtained by Pavlick and Kwiatkowski (2019) when re-annotating the SNLI dataset.”
Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair (2021, background): “378 Oversampling low-agreement examples is not 379 necessarily a bad thing if evaluated appropriately.380 Pavlick and Kwiatkowski (2019) and Nie et al.381 (2020c) show that there can be genuine disagree-382 ment between annotators over the example label,383 and argue that we should go beyond…”
When Combating Hype, Proceed with Caution (2021, background): “…objectives, once they become sufﬁciently capable, have a predictable reason to take on dangerous subgoals — like accumulating large amounts of computational, economic, or political power—to maximize the odds that their primary objectives are achieved (Bostrom, 2003; Omohundro, 2008; Bostrom, 2012).”