UnNatural Language Inference

By Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams. 2020

102 citations recorded by the source corpus.

Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to understand human-like syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle to understand the meaning of ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word order invariant. For example, in MNLI dataset we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists in pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Reasoning Like Program Executors (2022, background): “Meanwhile, neural models are notorious for their reliance on correlations among concrete tokens of a representation system and are usually assumed to be hard to grasp abstract rules of a symbolic reasoner (Helwe et al., 2021; Sinha et al., 2021).”
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? (2021, positive): “Our main argument throughout the paper shares the same logic as a recent line of studies (Sinha et al., 2021; O’Connor and Andreas, 2021; Pham et al., 2021; Gupta et al., 2021) which argue that the fact that LMs achieve good performance under ideal conditions is insufficient to establish language un”
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length (2021, methodology): “Thus, causal words do not appear particularly useful for GLUE.”
Out of Order: How important is the sequential order of words in a sentence in Natural Language Understanding tasks? (2020, background): “, 2019), natural language inference (Parikh et al., 2016; Sinha et al., 2020), and essay scoring (Parekh et al.”
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little (2021, methodology): “Sensitivity to word order for NLU tasks has recently been inves- tigated concurrently by Sinha et al. (2020), Pham et al. (2020), and Gupta et al. (2021).”
ANLIzing the Adversarial Natural Language Inference Dataset (2020, background): “Specific to NLI, some have probed models to see what they learn (Richardson et al., 2019; Sinha et al., 2021b), honed data col-”
Post-hoc Interpretability for Neural NLP: A Survey (2021, background): “A recent work, by Clouatre et al. [29], Sinha et al. [107], find that destroying syntax by shuffling words does not significantly affect a model trained on an NLI task, indicating that the model does not achieve natural language understanding.”
Sometimes We Want Ungrammatical Translations (2021, methodology): “(Si et al., 2019; Sinha et al., 2020; Pham et al., 2020; Gupta et al., 2021; Sinha et al., 2021).”
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision (2021, background): “While humans may have trouble understanding language without syntax, language models seem to be robust to text without syntax [41, 42].”
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality (2022, background): “Other work in this space has tried to understand what models know about syntax [24,28,34,49,54,71,83] or the complex interaction between syntactic and semantic categories [38,78,81,82].”
When does pretraining help?: assessing self-supervised learning for law and the CaseHOLD dataset of 53,000+ legal holdings (2021, background): “Additionally, BERT models are known to disregard word order [42].”