On Faithfulness and Factuality in Abstractive Summarization

By Joshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan T. McDonald. 2020

1,585 citations recorded by the source corpus.

It is well known that the standard likelihood training and approximate decoding objectives in neural text generation models lead to less human-like responses for open-ended tasks such as language modeling and story generation. In this paper we have analyzed limitations of these models for abstractive document summarization and found that these models are highly prone to hallucinate content that is unfaithful to the input document. We conducted a large scale human evaluation of several neural abstractive summarization systems to better understand the types of hallucinations they produce. Our human annotators found substantial amounts of hallucinated content in all model generated summaries. However, our analysis does show that pretrained models are better summarizers not only in terms of raw metrics, i.e., ROUGE, but also in generating faithful and factual summaries as evaluated by humans. Furthermore, we show that textual entailment measures better correlate with faithfulness than standard metrics, potentially leading the way to automatic evaluation metrics as well as training and decoding criteria.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “…generated by large language models are coherent or factually correct, as underscored by the recent work evaluating the factuality of language model generations and explanations (Maynez et al., 2020; Rashkin et al., 2021; Ye and Durrett, 2022; Marasovi ´ c et al., 2022; Wiegreffe et al., 2022).”
Toolformer: Language Models Can Teach Themselves to Use Tools (2023, background): “…include an inability to access up-to-date information on recent events (Komeili et al., 2022) and the related tendency to hallucinate facts (Maynez et al., 2020; Ji et al., 2022), difficulties in understanding low-resource languages (Lin et al., 2021), a lack of mathematical skills to…”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “…factuality, but identify questionanswering metrics as promising, somewhat contradicting Maynez et al.. Similarly, Chen et al. (2021) analyze mispredictions on a set of previously annotated summarization corpora (Kryscinski et al., 2020; Wang et al., 2020; Falke et al., 2019; Maynez et al., 2020).”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, methodology): “Since ROUGE only tests the extent to which a generated summary has a lexical overlap with a reference summary, it can erroneously produce high scores for fluent, yet meaningless and unfaithful outputs as long as many of the same words are used (Maynez et al., 2020; Gabriel et al., 2020).”
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023, background): “, 2019), generate non-factual information (Maynez et al., 2020; Longpre et al., 2021; Devaraj et al., 2022), and other challenges in deployment and evaluation (Zellers et al.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “works have outlined these risks (Maynez et al., 2020; Sheng et al., 2021; Weidinger et al., 2022; Zhuo et al., 2023), proposed taxonomies (Weidinger et al.”
WebGPT: Browser-assisted question-answering with human feedback (2021, background): “These are false statements that are the result of the model failing to achieve its training objective, including most hallucinations, which are statements that are false, but look plausible at a glance [Maynez et al., 2020].”
Learning to summarize from human feedback (2020, background): “making up facts [41]) and unimportant errors (e.”
Internet-augmented language models through few-shot prompting for open-domain question answering (2022, background): “Despite these successes, few-shot language models still lack a key ingredient; they are susceptible to hallucinations (Maynez et al., 2020) caused by incorrect retrieval of knowledge stored in their weights or due to the model having incomplete or outdated knowledge.”
You Only Need One Model for Open-domain Question Answering (2021, background): “More importantly however, these language models often suffer from Hallucinations (Shuster et al., 2021; Maynez et al., 2020; Zhou et al., 2021; Zellers et al., 2019; Roller et al., 2021; Mielke et al., 2020), generating factually incorrect but plausiblesounding outputs.”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “Hallucination in text-generation models is a topic that has received attention recently, particularly in the settings of summarization (Maynez et al., 2020), machine translation (Zhou et al.”
Aligning Language Models with Preferences through f-divergence Minimization (2023, background): “, 2020) language model to generate more factually faithful summaries (Maynez et al., 2020; Nan et al., 2021).”
Measuring Attribution in Natural Language Generation Models (2021, background): “Maynez et al. (2020) presented an extensive characterization of hallucinations, discussed behavior of models that generate content that are present in larger corpora beyond a given source and conducted a significant human study.”
PEER: A Collaborative Language Model (2022, background): “Beyond that, they are hard to control (Korbak et al., 2022) and verifying their outputs is challenging as they often hallucinate content (Maynez et al., 2020; Shuster et al., 2021; Nakano et al., 2021) and lack the ability to explain their intentions.”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, background): “NLI has broad applicability in NLP: it has proven useful for pretraining (Clark et al., 2019; Phang et al., 2018), and can be applied to verify candidate answers in question-answering (Chen et al., 2021) or factuality of generated summaries (Maynez et al., 2020).”
Evaluating the Factual Consistency of Abstractive Text Summarization (2019, methodology): “Table 3: Performance of models evaluated by means of weighted (class-balanced) accuracy and F1 score on the manually annotated test set of CNN/DailyMail (this work) and XSum (Maynez et al., 2020).”
QuestEval: Summarization Asks for Fact-based Evaluation (2021, background): “Several works have explored using natural language inference (NLI) models to evaluate the factual consistency of summaries (Kry´sci´nski et al., 2019; Falke et al., 2019; Maynez et al., 2020), ﬁnding mixed results in using NLI models rather than QA models.”