Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

By R. Thomas McCoy, Ellie Pavlick, Tal Linzen. 2019

1,462 citations recorded by the source corpus.

A machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases. We study this issue within natural language inference (NLI), the task of determining whether one sentence entails another. We hypothesize that statistical NLI models may adopt three fallible syntactic heuristics: the lexical overlap heuristic, the subsequence heuristic, and the constituent heuristic. To determine whether models have adopted these heuristics, we introduce a controlled evaluation set called HANS (Heuristic Analysis for NLI Systems), which contains many examples where the heuristics fail. We find that models trained on MNLI, including BERT, a state-of-the-art model, perform very poorly on HANS, suggesting that they have indeed adopted these heuristics. We conclude that there is substantial room for improvement in NLI systems, and that the HANS dataset can motivate and measure progress in this area.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Language Models are Few-Shot Learners (2020, background): “The main disadvantages are the need for a new large dataset for every task, the potential for poor generalization out-of-distribution [MPL19], and the potential to exploit spurious features of the training data [GSL + 18, NK19].”
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, methodology): “…systems: when tested on cases sampled from a distribution that differs from the one they were trained on, their behavior is unpredictable and inconsistent with that of humans (Jia and Liang, 2017; McCoy et al., 2019b), and they require extensive instruction on each new task (Yogatama et al., 2019).”
Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “, 2018) and certain superficial syntactic properties (McCoy et al., 2019).”
Probing Neural Network Comprehension of Natural Language Arguments (2019, background): “There is much recent work focusing on statistical cues in datasets in vision (Jo and Bengio, 2017) and NLP (Sanchez et al., 2018; McCoy et al., 2019; Gururangan et al., 2018; Glockner et al., 2018; Poliak et al., 2018; Rajpurkar et al., 2018; Jia and Liang, 2017).”
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, neutral): “Much recent work has made similar observations about the limitations of existing pretrained models (Jia and Liang, 2017; Naik et al., 2018; McCoy and Linzen, 2019; McCoy et al., 2019; Liu et al., 2019a,b).”
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (2019, background): “For instance, evidence suggests (Gururangan et al., 2018; Jia and Liang, 2017; McCoy et al., 2019) that simply asking human annotators to write candidate statements will result in examples that typically only require surface-level reasoning.”
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, background): “HANS Heuristic Analysis for NLI Systems (HANS) [40] is an NLI dataset designed to evaluate the tendency of models to exploit fallible, superficial syntactic heuristics in NLP data.”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “However, interpretability methods are a useful tool for evaluations, since they may be used to uncover model shortcuts (McCoy et al., 2019) or to find systematic errors (Popovic and Ney, 2011) that would be described in an evaluation report.17 Interpretability tools can also facilitate evaluations.”
Shortcut learning in deep neural networks (2020, background): “Feature combination crucially depends on shortcut features like word length [59, 60, 14, 61], and consequently leads to a severe lack of robustness such as an inability to generalise to more challenging test conditions [62, 63, 64, 65].”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Further, as with many black-box models, coarse classifiers overfit to spurious artifacts (Gururangan et al., 2018; McCoy et al., 2019; Kumar et al., 2019) and amplify biases in their training data (Zhao et al., 2017; Sun et al., 2019).”
Language Models as Knowledge Bases? (2019, background): “More recently, McCoy et al. (2019) found that for natural language inference, a model based on BERT learns to rely heavily on fallible syntactic heuristics instead of a deeper understanding of the natural language input.”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, methodology): “We connect our results to previous work that reported instability in performance on stress tests designed to diagnose shortcut learning in Natural Language Inference tasks (McCoy et al., 2019b; Naik et al., 2018).”
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, background): “There has been ample evidence that such models overﬁt to spurious artifacts (Gu-rurangan et al., 2018; McCoy et al., 2019; Kumar et al., 2019) and amplify biases in data (Zhao et al., 2017; Sun et al., 2019).”
Measuring Association Between Labels and Free-Text Rationales (2020, background): “These, along with behavioral tests such as challenge sets (McCoy et al., 2019) and checklists (Ribeiro et al., Although gradient-attribution has been extensively studied in NLP, its interplay with free-text rationalization has not.”
Discovering Language Model Behaviors with Model-Written Evaluations (2022, methodology): “Other work automatically constructs evaluations not with LMs but using example templates or programmatic generation (Weston et al., 2016; Johnson et al., 2017; McCoy et al., 2019).”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, background): “For crowdsourcing NLI datasets, where the anno-tator is given a premise and asked to write a hypothesis of each label (Bowman et al., 2015; Williams et al., 2018), the presence of annotation artifacts is especially well-studied (Gururangan et al., 2018; McCoy et al., 2019; Glockner et al., 2018).”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “…is known to result in machine-exploitable annotator artefacts (Jia and Liang, 2017; Schwartz et al., 2017; Gururangan et al., 2018; Geva et al., 2019), leading to poor out-of-distribution generalisation (Chen et al., 2016; Weissenborn et al., 2017; Yogatama et al., 2019; McCoy et al., 2019).”
Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “9 16 El Grecos contain this collection) improves performance on HANS, a challenge set for diagnosing fallible syntactic heuristics in NLI models (McCoy et al., 2019).”