SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders

By Peter J. Liu, Yu-An Chung, Jie Jessie Ren. 2019

20 citations recorded by the source corpus.

We propose an end-to-end neural model for zero-shot abstractive text summarization of paragraphs, and introduce a benchmark task, ROCSumm, based on ROCStories, a subset for which we collected human summaries. In this task, five-sentence stories (paragraphs) are summarized with one sentence, using human summaries only for evaluation. We show results for extractive and human baselines to demonstrate a large abstractive gap in performance. Our model, SummAE, consists of a denoising auto-encoder that embeds sentences and paragraphs in a common space, from which either can be decoded. Summaries for paragraphs are generated by decoding a sentence from the paragraph representations. We find that traditional sequence-to-sequence auto-encoders fail to produce good summaries and describe how specific architectural choices and pre-training techniques can significantly improve performance, outperforming extractive baselines. The data, training, evaluation code, and best model weights are open-sourced.

Subfields: Artificial Intelligence, Natural Language Processing, Summarization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, methodology): “However, the t-SNE visualization of the latent space in Liu et al. (2019b) allowed us to assume that encoders can output a latent representation z , attending to content rather than on an attribute, with a similar training.”
AgreeSum: Agreement-Oriented Multi-Document Summarization (2021, background): “Liu et al. (2019a) improve the model by encoding articles and summaries in the same space.”