Enhancing Factual Consistency of Abstractive Summarization

By Chenguang Zhu, William Fu-Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, Meng Jiang. 2021

158 citations recorded by the source corpus.

Automatic abstractive summaries are found to often distort or fabricate facts in the article. This inconsistency between summary and original text has seriously impacted its applicability. We propose a fact-aware summarization model FASum to extract and integrate factual relations into the summary generation process via graph attention. We then design a factual corrector model FC to automatically correct factual errors from summaries generated by existing systems. Empirical results show that the fact-aware summarization can produce abstractive summaries with higher factual consistency compared with existing systems, and the correction model improves the factual consistency of given summaries via modifying only a few keywords.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “between entities (Nan et al., 2021; Zhu et al., 2021), (2) improving coreference to mitigate gender bias in translation (Levy et al.”
Survey of Hallucination in Natural Language Generation (2022, methodology): “[240] propose to use an explicit graph neural network (GNN) to encode the fact tuples extracted from source documents.”
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation (2020, unknown): “Human evaluations of similar sizes (50/100/109/10 samples) are conducted by previous studies (Dong et al., 2020; Zhu et al., 2021; Matsumaru et al., 2020; Nan et al., 2021).”
Don’t Say What You Don’t Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search (2022, methodology): “Two popular approaches are 1) incorporating extracted knowledge (Zhu et al., 2021)”
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization (2021, methodology): “…for deploying these systems in real-460 world scenarios, as such recent work has studied461 methods to improve the faithfulness of abstractive462 summarization systems (Zhao et al., 2020; Dong463 et al., 2020; Goyal and Durrett, 2021; Xu et al.,464 2020; Chen et al., 2021; Zhu et al., 2021).”
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2023, background): “tual consistency, including applying attentions to fact triples extracted from source documents (Cao et al., 2018; Zhu et al., 2021), optimizing summarization models towards a factual consistency metrics (Nan et al.”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “Another common practice involves constructing an auxiliary fixer to rectify hallucinations during the post-processing stage (Cao et al., 2020; Zhu et al., 2021; Fabbri et al., 2022).”
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021, methodology): “(1) Employ information extraction tools to extract facts and leveraging it by building additional ob-jective (Cao et al., 2018; Goodrich et al., 2019; Zhang et al., 2020a; Zhu et al., 2021).”
Factual Error Correction for Abstractive Summaries Using Entity Retrieval (2022, unknown): “Hence, to train a system, we construct a synthetic dataset by modifying the reference summaries following previous works (Cao et al., 2020; Zhu et al., 2021; Kryscinski et al., 2020).”
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization (2021, background): “Three types of remedies have been proposed: running a separately learned error correction component (Dong et al., 2020), removing noisy training samples (Nan et al., 2021; Goyal and Durrett, 2021), and modifying the Transformer architecture (Huang et al., 2020; Zhu et al., 2021).”