Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation

By Yuning Mao, Xiang Ren, Heng Ji, Jiawei Han. 2020

41 citations recorded by the source corpus.

Summaries generated by abstractive summarization are supposed to only contain statements entailed by the source documents. However, state-of-the-art abstractive methods are still prone to hallucinate content inconsistent with the source documents. In this paper, we propose constrained abstractive summarization (CAS), a general setup that preserves the factual consistency of abstractive summarization by specifying tokens as constraints that must be present in the summary. We explore the feasibility of using lexically constrained decoding, a technique applicable to any abstractive method with beam search decoding, to fulfill CAS and conduct experiments in two scenarios: (1) Standard summarization without human involvement, where keyphrase extraction is used to extract constraints from source documents; (2) Interactive summarization with human feedback, which is simulated by taking missing tokens in the reference summaries as constraints. Automatic and human evaluations on two benchmark datasets demonstrate that CAS improves the quality of abstractive summaries, especially on factual consistency. In particular, we observe up to 11.2 ROUGE-2 gains when several ground-truth tokens are used as constraints in the interactive summarization scenario.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “and to encourage models to generate more factual text (Mao et al., 2020; Stiennon et al., 2020).”
Planning with Learned Entity Prompts for Abstractive Summarization (2021, methodology): “, 2021) or (ii) used to filter beams for lexically constrained decoding (Mao et al., 2020), to control the summary generation.”
Annotating and Modeling Fine-grained Factuality in Summarization (2021, methodology): “Recent work has explored different methods for enforcing factuality: modifying the model, such as encoding SRL structures in the input (Cao et al., 2018), post-hoc correction (Dong et al., 2020), or constrained decoding (Song et al., 2020a; Mao et al., 2020).”
Don’t Say What You Don’t Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search (2022, methodology): “…already-generated summaries (Dong et al., 2020; Zhu et al., 2021), using a knowledge graph (Zhu et al., 2021), filtering training data (Nan et al., 2021), constraining generation with keywords (Mao et al., 2020), using NLI models (Barrantes et al., 2020; Mishra et al., 2020), among others.”
InFillmore: Frame-Guided Language Generation with Bidirectional Context (2021, methodology): “, 2021), and abstractive summarization (Mao et al., 2020).”