Controlling the Amount of Verbatim Copying in Abstractive Summarization

By Kaiqiang Song, Bingqing Wang, Zhe Feng, Liu Ren, Fei Liu. 2019

54 citations recorded by the source corpus.

An abstract must not change the meaning of the original text. A single most effective way to achieve that is to increase the amount of copying while still allowing for text abstraction. Human editors can usually exercise control over copying, resulting in summaries that are more extractive than abstractive, or vice versa. However, it remains poorly understood whether modern neural abstractive summarizers can provide the same flexibility, i.e., learning from single reference summaries to generate multiple summary hypotheses with varying degrees of copying. In this paper, we present a neural summarization model that, by learning from single human abstracts, can produce a broad spectrum of summaries ranging from purely extractive to highly generative ones. We frame the task of summarization as language modeling and exploit alternative mechanisms to generate summary hypotheses. Our method allows for control over copying during both training and decoding stages of a neural summarization model. Through extensive experiments we illustrate the significance of our proposed method on controlling the amount of verbatim copying and achieve competitive results over strong baselines. Our analysis further reveals interesting and unobvious facts.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, background): “Although there has been a tremendous amount of progress in abstractive summarization in recent years, most research has focused on monolingual summarization because of the lack of high quality multilingual resources (Lewis et al., 2019a; Song et al., 2020).”
Annotating and Modeling Fine-grained Factuality in Summarization (2021, methodology): “A similar objective has been used by prior work (Song et al., 2020b) to encourage the model to copy words present in the source.”
How well do you know your summarization datasets? (2021, unknown): “…(2020) 4 : (1) H ETER G RAPH (Wang et al., 2020), (2) M ATCH S UMM (Lewis et al., 2020), (3) R EFRESH (Narayan et al., 2018b) , (4) T WO S TAGE RL (Song et al., 2020), (5) N EUSUMM (Wang et al., 2019) , (6) B OT - TOM U P (Gehrmann et al., 2018) (Lewis et al., 2020) For XSum, we use the outputs…”
Joint Parsing and Generation for Abstractive Summarization (2019, background): “2018; Lebanoff, Song, and Liu 2018; Gehrmann, Deng, and Rush 2018), convolutional networks (Chopra, Auli, and Rush 2016; Narayan, Cohen, and Lapata 2018), or Transformer (Devlin et al. 2018; Liu et al. 2018; Song et al. 2020).”
Don’t Say What You Don’t Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search (2022, background): “Some works control the extractiveness of generations (Song et al., 2020).”