The Summary Loop: Learning to Write Abstractive Summaries Without Examples

By Philippe Laban, Andrew Hsi Bloomberg, John F. Canny, Marti A. Hearst. 2020

60 citations recorded by the source corpus.

This work presents a new approach to unsupervised abstractive summarization based on maximizing a combination of coverage and fluency for a given length constraint. It introduces a novel method that encourages the inclusion of key terms from the original document into the summary: key terms are masked out of the original document and must be filled in by a coverage model using the current generated summary. A novel unsupervised training procedure leverages this coverage model along with a fluency model to generate and score summaries. When tested on popular news summarization datasets, the method outperforms previous unsupervised methods by more than 2 R-1 points, and approaches results of competitive supervised methods. Our model attains higher levels of abstraction with copied passages roughly two times shorter than prior work, and learns to compress and merge sentences without supervision.

Subfields: Artificial Intelligence, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Generative Models

Citations in the corpus

Few-Shot Text Generation with Natural Language Instructions (2020, background): “…NLP commonly require large sets of examples from re-lated tasks (Gu et al., 2018; Dou et al., 2019; Qian and Yu, 2019; Ye et al., 2020), parallel data for consistency training (Xie et al., 2020; Chen et al., 2020), or highly specialized methods tailored towards a speciﬁc task (Laban et al., 2020).”
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2020, methodology): “Laban et al. (2020) train an unsupervised summarization model by guiding the model with reinforcement learning rewards.”
Few-Shot Text Generation with Pattern-Exploiting Training (2020, background): “…learning in NLP commonly require large sets of examples from re-lated tasks (Gu et al., 2018; Dou et al., 2019; Qian and Yu, 2019), parallel data for consistency training (Xie et al., 2019; Chen et al., 2020), or highly specialized methods tailored towards a speciﬁc task (Laban et al., 2020).”