Probing Neural Network Comprehension of Natural Language Arguments

By Timothy Niven, Hung-Yu kao. 2019

498 citations recorded by the source corpus.

We are surprised to find that BERT’s peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models achieve random accuracy. Our adversarial dataset provides a more robust assessment of argument comprehension and should be adopted as the standard in future work.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Language Models are Few-Shot Learners (2020, background): “The main disadvantages are the need for a new large dataset for every task, the potential for poor generalization out-of-distribution [MPL19], and the potential to exploit spurious features of the training data [GSL + 18, NK19].”
Automatic Detection of Generated Text is Easiest when Humans are Fooled (2019, background): “It is worth noting that recent work has raised questions as to whether BERT truly builds a semantic understanding to make its predictions, or whether it merely takes advantage of spurious statistical differences between the text of different classes (Niven and Kao, 2019).”
A Hybrid Neural Network Model for Commonsense Reasoning (2019, methodology): “In future work, we plan to extend HNN to more sophisticated reasoning tasks, especially those where large-scale language models like BERT and GPT do not perform well, as discussed in (Gao et al., 2019; Niven and Kao, 2019).”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “Heinzerling [2019], in a discussion inspired by Niven and Kao [2019], suggests various data ablations that can be applied to test data as well, to investigate what cues a system might be relying on.”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “…if there are systematic gaps and biases in training data, models may perform deceptively well on corresponding held-out test sets by learning simple decision rules rather than encoding a more generalisable understanding of the task (e.g. Niven and Kao, 2019; Geva et al., 2019; Shah et al., 2020).”
Shortcut learning in deep neural networks (2020, background): “Attempts like incorporating a certain degree of unsupervised training as employed in prominent language models like BERT [5] and GPT-2 [66] did not resolve the problem of shortcut learning [14].”
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, background): “5 łReasoningž in NLP There is still a debate(7) about if deep neural networks are capable of performing łreasoningž or just memorizing patterns based on large training data [3, 89].”
On Faithfulness and Factuality in Abstractive Summarization (2020, background): “This suggests that while some studies argue that large-scale pretrained models are merely better at learning data-speciﬁc regularities (Niven and Kao, 2019), at least on in-domain summarization the gains in automatic metrics are realized in observable differences by humans.”
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (2020, background): “As noted in § 2, both McCoy et al. (2019) and Niven and Kao (2019) found that BERT picked up idiosyncratic patterns in the data for their tasks, and not “mean-ing”.”
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020, background): “Nevertheless, the presence of these gaps is well-documented (Gururangan et al., 2018; Poliak et al., 2018; Min et al., 2019), and Niven and Kao (2019) give an initial attempt at formally characterizing them.”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, background): “However, the strong performance of the Transformer comes at the cost of interpretability, and this lack of transparency can hide underlying problems such as model bias and spurious correlations [41, 55, 75].”
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2020, background): “Not only have fine-tuned models been shown to pick up spurious patterns and biases present in the training data (Niven and Kao, 2019; McCoy et al., 2019), but also to exhibit a large training instability: fine-tuning a model multiple times on the same dataset, varying only the random seed, leads to ”
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021, background): “• An evaluation dataset should be sufﬁciently free of annotation artifacts (as in Si et al., 2019; Sugawara et al., 2020b; Niven and Kao, 2019) that a system cannot reach near-human levels of performance by any means other than demonstrating the required language-related behaviors.”
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models (2020, background): “ authors proposed that the visualization of loss landscapes and optimization trajectories in BERT netuning stage can help people understand the eectiveness of BERT. Interestingly, it was observed in [25] that the exploitation of suspicious statistical cues in the dataset is accounted for the strong ”
When BERT Plays the Lottery, All Tickets Are Winning (2020, background): “However, we have ample evidence that BERT is very adapt at learning not just the new tasks, but also all kinds of biases present in the task-speciﬁc data (McCoy et al., 2019; Rogers et al., 2020a; Jin et al., 2020; Niven and Kao, 2019; Zellers et al., 2019).”