Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text

By Sebastian Gehrmann, Elizabeth Clark, Thibault Sellam. 2022

245 citations recorded by the source corpus.

Evaluation practices in natural language generation (NLG) have many known flaws, but improved evaluation approaches are rarely widely adopted. This issue has become more urgent, since neural generation models have improved to the point where their outputs can often no longer be distinguished based on the surface-level features that older metrics rely on. This paper surveys the issues with human and automatic model evaluations and with commonly used datasets in NLG that have been pointed out over the past 20 years. We summarize, categorize, and discuss how researchers have been addressing these issues and what their findings mean for the current state of model evaluations. Building on those insights, we lay out a long-term vision for evaluation research and propose concrete steps for researchers to improve their evaluation processes. Finally, we analyze 66 generation papers from recent NLP conferences in how well they already follow these suggestions and identify which areas require more drastic changes to the status quo.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, methodology): “, tasks from GEM (Gehrmann et al., 2021), and TyDiQA (Clark et al., 2020).”
Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “Human evaluation, while a gold standard, can be complicated by various HCI considerations (Clark et al., 2021; Gehrmann et al., 2023), and is not always scalable.”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “Following the suggestions by Gehrmann et al. (2022b), we report ROUGE-2, ROUGE-L (Lin, 2004),31 and Levenshtein distance.”
UL2: Unifying Language Learning Paradigms (2022, methodology): “For generation tasks, we generally report ROUGE-2 following the advice of (Gehrmann et al., 2022).”
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation (2022, methodology): “Of the 39 papers that conducted human evaluation, none of the papers did power analysis, the same as the result provided in the previous survey of Gehrmann et al. (2022b).”
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022, methodology): “racy and instead, NLG relies on automatic and human evaluation approaches that constantly improve (Gehrmann et al., 2022; Kasai et al., 2022).”
News Summarization and Evaluation in the Era of GPT-3 (2022, methodology): “In our work, we chose a human evaluation workflow that directly asks annotators to compare systems, while other prior work has opted for Likert-scale judgments and/or evaluation along multiple quality dimensions (Gehrmann et al., 2022).”
BloombergGPT: A Large Language Model for Finance (2023, background): “LLM evaluation remains a challenging and evolving problem (Gehrmann et al., 2022; Goyal et al., 2022), with new benchmarks trying to standardize evaluation”
Unifying Language Learning Paradigms (2022, methodology): “For generation tasks, we generally report ROUGE-2 following the advice of (Gehrmann et al., 2022).”
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way (2022, background): “…the domain of the data poses a signi  cant challenge for system evaluation: Automatic evaluation metrics for summarization are unreliable (Kryscinski et al., 2019; Gehrmann et al., 2022), but the summaries are too technical and jar-gonistic for non-specialist human raters to evaluate reliably.”
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark (2021, unknown): “1162 /t a c l _a _ 00506 / 2043760 /t a c l _a _ 00506 . pd f b y gue s t on 25 O c t obe r 2022 language generation (NLG) evaluation metrics (Gehrmann et al., 2022, 2021), for various tasks including machine translation (Freitag et al., 2021; Mathur et al., 2020), data-to-text generation (Dhingra…”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “…about lack of standardization, reproducibility, validity, and generalizability to real-world settings (Belz et al., 2021; Clark et al., 2021; Gehrmann et al., 2023; Howcroft et al., 2020). d For example, BIG-bench (Srivastava et al., 2022) consists of more than 200 language tasks…”
TaTa: A Multilingual Table-to-Text Dataset for African Languages (2022, background): “Human Evaluation Automatic metrics are untested for many of our languages and the setting of T A TA, and outputs may still be correct without matching references (Gehrmann et al., 2022).”
RankGen: Improving Text Generation with Large Ranking Models (2022, background): “human evaluation remains critical for open-ended generation (Celikyilmaz et al., 2020; Gehrmann et al., 2022).”
SafeText: A Benchmark for Exploring Physical Safety in Language Models (2022, unknown): “(Xiao and Wang, 2021; Gehrmann et al., 2022; Ji et al., 2022).”
Dialect-robust Evaluation of Generated Text (2022, background): “For example, machine translation systems are usually evaluated by whether they match references in the target language, for which the dialect is generally unspecified (Gehrmann et al., 2022).”
Towards a Unified Multi-Dimensional Evaluator for Text Generation (2022, background): “These metrics are potentially misleading as NLG models have advanced to the point where discrepancies between them are unlikely to be detected based on surface-level features (Gehrmann et al., 2022).”