All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text

By Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, Noah A. Smith. 2021

548 citations recorded by the source corpus.

Human evaluations are typically considered the gold standard in natural language generation, but as models’ fluency improves, how well can evaluators detect and judge machine-generated text? We run a study assessing non-experts’ ability to distinguish between human- and machine-authored text (GPT2 and GPT3) in three domains (stories, news articles, and recipes). We find that, without training, evaluators distinguished between GPT3- and human-authored text at random chance level. We explore three approaches for quickly training evaluators to better identify GPT3-authored text (detailed instructions, annotated examples, and paired examples) and find that while evaluators’ accuracy improved up to 55%, it did not significantly improve across the three domains. Given the inconsistent results across text domains and the often contradictory reasons evaluators gave for their judgments, we examine the role untrained human evaluations play in NLG evaluation and provide recommendations to NLG researchers for improving human evaluations of text generated from state-of-the-art models.

Subfields: Artificial Intelligence, Information Systems, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Evaluation & Benchmarks

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Clark et al. (2021) find MTurk evaluators are more likely to base judgments of generated text on the text’s form rather than its content.”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “To create this dataset, we leverage the massive pretrained language model GPT-3 (Brown et al., 2020), which is known to produce close-to-human-like text (Clark et al., 2021; Dou et al., 2021) but also easily generates socially biased and toxic content (Sheng et al., 2019; Gehman et al., 2020).”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, background): “Despite known deficiencies such as incoherence or repetition (Dou et al., 2021), these models often produce human-like text (Clark et al., 2021) and show potential for creative writing tasks (Lee et al., 2022).”
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022, background): “have proven very impressive at generating humanlike language for conversational, summarization, or sentence continuation settings, often with zero to few examples to learn from (Brown et al., 2020; Clark et al., 2021; Chowdhery et al., 2022).”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, background): “They provide an effective interface for few-shot learning [8], show impressive natural-language understanding capabilities [47], and, in some contexts, their generations can be indistinguishable from human-authored text [11].”
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers (2021, background): “Concurrent work has shown that human evaluation might not always be consistent [10, 29]; however human judgments continue to be the gold standard for evaluating open-ended text generation.”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “…leading to critiques about lack of standardization, reproducibility, validity, and generalizability to real-world settings (Belz et al., 2021; Clark et al., 2021; Gehrmann et al., 2023; Howcroft et al., 2020). d For example, BIG-bench (Srivastava et al., 2022) consists of more than 200…”
RankGen: Improving Text Generation with Large Ranking Models (2022, methodology): “Note that we used the MTurk Sandbox interface only — no MTurk workers are recruited in our human study due to poor annotation quality for open-ended text generation (Karpinska et al., 2021; Clark et al., 2021).”
Can Large Language Models Be an Alternative to Human Evaluations? (2023, background): “While human evaluation is prevalent and indispensable in NLP, it is notoriously unstable (Gillick and Liu, 2010; Clark et al., 2021).”
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it (2022, background): “This seems particularly relevant considering that Clark et al. (2021) recently found that untrained crowdworkers, who serve as participants in the majority of human evaluation studies, cannot distinguish between stories written by humans and stories generated by GPT-3.”
Human heuristics for AI-generated language are flawed (2022, background): “However, the potential for improving human intuition for the detection of generated language is likely limited (16), and future generations of language models may invalidate learned heuristics.”
News Summarization and Evaluation in the Era of GPT-3 (2022, background): “This demonstrate two things: first, when comparing summaries from two strong models, the choice is inherently ambiguous (similar observations in Clark et al. (2021)).”
Do Language Models Plagiarize? (2022, background): “A distinction between machine-authored and human-written content has even become quite challenging (Clark et al., 2021; Uchendu et al., 2020).”
Aligning Large Language Models with Human: A Survey (2023, background): “In addition, the increasing quality of generated text from LLMs makes it more challenging for human annotators to distinguish between human-written and LLMgenerated text in the open-ended NLP tasks (Clark et al., 2021).”
Humans or LLMs as the Judge? A Study on Judgement Bias (2024, background): “Human judges are also found to have inherent bias (Zheng et al., 2023; Wu and Aji, 2023) and may not even provide reliable answers (Clark et al., 2021; Hämäläinen et al., 2023).”