Measuring Attribution in Natural Language Generation Models

By Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, David Reitter. 2021

277 citations recorded by the source corpus.

Abstract Large neural models have brought a new challenge to natural language generation (NLG): It has become imperative to ensure the safety and reliability of the output of models that generate freely. To this end, we present an evaluation framework, Attributable to Identified Sources (AIS), stipulating that NLG output pertaining to the external world is to be verified against an independent, provided source. We define AIS and a two-stage annotation pipeline for allowing annotators to evaluate model output according to annotation guidelines. We successfully validate this approach on generation datasets spanning three tasks (two conversational QA datasets, a summarization dataset, and a table-to-text dataset). We provide full annotation guidelines in the appendices and publicly release the annotated data at https://github.com/google-research-datasets/AIS.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “…generated by large language models are coherent or factually correct, as underscored by the recent work evaluating the factuality of language model generations and explanations (Maynez et al., 2020; Rashkin et al., 2021; Ye and Durrett, 2022; Marasovi ´ c et al., 2022; Wiegreffe et al., 2022).”
LaMDA: Language Models for Dialog Applications (2022, background): “The recently-proposed Attributable to Identified Sources (AIS) framework [88] articulates a more precise approach to assess output of language models that pertains to the external world.”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “For table-to-text, Thomson and Reiter (2020) report the performance of an information extraction-based metric (Wiseman et al.”
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023, background): “…the utterances generated by dialogue systems into three categories, i.e., fully attributable, not fully attributable, and generic; and the Attributable to Identified Sources (AIS) benchmark (Rashkin et al., 2021) assesses whether the source documents support the output of text generation models.”
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023, background): “Prior work in model-based evaluation mainly reports aggregated scores since the goal is a comparison between different systems being evaluated (Zhang et al., 2020; Rashkin et al., 2021; Gao et al., 2022) while we report both to see the relationship between two types of metrics.”
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022, methodology): “A large commonality in the majority of prior work seeks to address hallucination by ameliorating the model (Shuster et al., 2021; Mielke et al., 2020; Dziri et al., 2021a; Rashkin et al., 2021b), but no attempt has been made so far to audit the conversational benchmarks to the best of our knowledge.”
LiveBench: A Challenging, Contamination-Free LLM Benchmark (2024, background): “Whenever humans are involved in annotation (of which judging is a sub-case), design choices or factors can cause high error rates [32], and even in well-designed human-annotation setups, high variability from human to human leads to unpredictable outcomes [44].”
“According to . . . ”: Prompting Language Models Improves Quoting from Pre-Training Data (2023, background): “A related line of work is attribution of generated text to their sources (Rashkin et al., 2021; Bohnet et al., 2022).”
Enabling Large Language Models to Generate Text with Citations (2023, methodology): “The NLI evaluation is in accordance with the attributable to identified sources (AIS) framework (Rashkin et al., 2021): φ(concat(Ci), si) = 1 implies that si is true based solely on concat(Ci).”
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark (2021, background): “This is consistent with recent analyses on WOW, CMU-DOG, and TOPICALCHAT which revealed that more than 60% of the ground-truth responses are unattributable to the knowledge (Dziri et al., 2022b; Rashkin et al., 2021a).”
Chain-of-Verification Reduces Hallucination in Large Language Models (2023, methodology): “Other approaches include using tools for fact-checking (Chern et al., 2023a; Galitsky, 2023; Peng et al., 2023), or linking to external documents with attribution (Menick et al., 2022; Rashkin et al., 2023; Gao et al., 2023).”
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems (2023, background): “Second, we offer a novel development pipeline for fine-tuning lightweight LLMs on synthetically generated queries and answers.”
ExpertQA: Expert-Curated Questions and Attributed Answers (2023, methodology): “Notably, Rashkin et al. (2021) propose the framework of Attributable to Identified Sources (AIS) for performing human evaluation of attributions.”
TRUE: Re-evaluating Factual Consistency Evaluation (2022, methodology): “Finally, we encourage data curators to use the binary annotation scheme, which is inline with the recommendations of Rashkin et al. (2021a).”
Evaluating Verifiability in Generative Search Engines (2023, methodology): “To collect these binary judgments, we use the attributable to identified sources (AIS) evaluation framework of (Rashkin et al., 2022).”
RARR: Researching and Revising What Language Models Say, Using Language Models (2022, methodology): “Apart from human evaluation (Rashkin et al., 2021), several automated evaluation methods have been proposed.”
Attributed Text Generation via Post-hoc Research and Revision (2022, methodology): “To trust the output of a TGM, we would like each generation to be justiﬁed by an attribution report (Rashkin et al., 2021) that contains one or more pieces of supporting evidence from a trusted source (e.g., encyclopedia or articles) where appropriate.”