What Will it Take to Fix Benchmarking in Natural Language Understanding?

By Samuel R. Bowman, George E. Dahl. 2021

235 citations recorded by the source corpus.

Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and biased systems score so highly on standard benchmarks that there is little room for researchers who develop better systems to demonstrate their improvements. The recent trend to abandon IID benchmarks in favor of adversarially-constructed, out-of-distribution test sets ensures that current models will perform poorly, but ultimately only obscures the abilities that we want our benchmarks to measure. In this position paper, we lay out four criteria that we argue NLU benchmarks should meet. We argue most current benchmarks fail at these criteria, and that adversarial data collection does not meaningfully address the causes of these failures. Instead, restoring a healthy evaluation ecosystem will require significant progress in the design of benchmark datasets, the reliability with which they are annotated, their size, and the ways they handle social bias.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Additionally, treating datasets as dynamic encourages the development of evaluation suites that everyone can benefit from (Bowman and Dahl, 2021).”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “…2021, Malik, 2020] — however as Bowman and Dahl note, it remains a generally neglected issue as “[t]his criterion is difﬁcult to fully formalize, and we know of no simple test that would allow one to determine if a benchmark presents a valid measure of model ability” [Bowman and Dahl, 2021, p. 3].”
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey (2022, methodology): “We choose research agendas to ask about based on criticisms, debates, or findings in the literature and public sphere, for example regarding current practice in benchmarking (Bowman and Dahl, 2021; Raji et al., 2021), the relative value of advances in model architectures (Narang et al.”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, background): “As much as large-scale crowdsourced datasets have expedited progress on various NLP problems, a growing body of research has revealed fundamental limitations in existing datasets: they are often flooded with repetitive or spurious patterns, rather than covering the broad range of linguistic phenomen”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, methodology): “…et al., 2021; Wallace et al., 2021), although the approach has been challenged for not necessarily leading to better generalisation on non-adversarial test sets (Kaushik et al., 2021) and being sensitive to the choice of model that was used in the loop (Bowman and Dahl, 2021; Phang et al., 2021).”
GPQA: A Graduate-Level Google-Proof Q&A Benchmark (2023, neutral): “Furthermore, constructing such datasets using human annotators is difficult; As models continue to advance, evaluation datasets must meet a higher standard of quality in order to remain meaningful and not saturate (Bowman & Dahl, 2021).”
QuALITY: Question Answering with Long Input Texts, Yes! (2021, background): “versarial design would not be practical because model behavior would not provide enough signal to the crowdworkers, and we would risk limiting the usefulness of QuALITY as a test set for a full range of models (Bowman and Dahl, 2021).”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, background): “It could cause annotators to write unnatural 055 examples or to focus on a narrow subset of unusual 056 examples that models find difficult to learn, thus de- 057 creasing data diversity (Bowman and Dahl, 2021).”
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022, background): “Furthermore, this process ignores properties that do not match the leaderboard metric (Ethayarajh and Ju-rafsky, 2020; Bowman and Dahl, 2021; Dehghani et al., 2021).”
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets (2021, background): “For these reasons, many in the machine learning community have begun to call for research into new dataset collection methods [9, 38].”
CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge (2021, methodology): “Even without resorting to adversarial filtering, which artificially biases a dataset against existing model checkpoints [Bowman and Dahl, 2021], we find our annotation protocol leads to challenging claims for existing models.”
When Combating Hype, Proceed with Caution (2021, methodology): “8 In practice, research into more effective crowdsourcing and benchmark design (Kiela et al., 2021; Bowman and Dahl, 2021; Nangia et al., 2021; Gehrmann et al., 2021) and research into better statistical reporting and publication norms (Dodge et al.”
Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (2021, background): “Several researchers have long articulated concern for the correctness of data labels as 141 an indicator of internal validity [17, 105].”
Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair (2021, background): “052 However, one potential issue is that an adversari- 053 ally constructed dataset that targets a specific model 054 may bias the resulting data, creating datasets that 055 are unduly challenging for one class of models but 056 not others (Bowman and Dahl, 2021).”
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little (2021, background): “…perhaps subsequently acknowledge that we may not need human-like linguistic abilities for most NLP tasks, or alternatively, we need harder and more comprehensive evaluations (cf. Bowman and Dahl 2021), if we genuinely want to measure linguistic abilities, however those are defined, in our models.”