SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents

By Ramesh Nallapati, Feifei Zhai, Bowen Zhou. 2016

1,337 citations recorded by the source corpus.

We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art. Our model has the additional advantage of being very interpretable, since it allows visualization of its predictions broken up by abstract features such as information content, salience and novelty. Another novel contribution of our work is abstractive training of our extractive model that can train on human generated reference summaries alone, eliminating the need for sentence-level extractive labels.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Deep Reinforced Model for Abstractive Summarization (2017, background): “Most summarization models studied in the past are extractive in nature (Neto et al., 2002; Dorr et al., 2003; Filippova and Altun, 2013; Col-menares et al., 2015; Nallapati et al., 2017 A well-studied set of summarization tasks is the Document Understanding Conference (DUC) .”
Fine-tune BERT for Extractive Summarization (2019, methodology): “Although many neural models have been proposed for extractive summarization recently (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018; Dong et al., 2018; Zhang et al., 2018; Zhou et al., 2018), the improvement on automatic metrics like ROUGE has reached a bottleneck due to the…”
Get To The Point: Summarization with Pointer-Generator Networks (2017, methodology): “However, our best model does not quite surpass the ROUGE scores of the lead-3 baseline, nor the current best extractive model (Nallapati et al., 2017).”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, neutral): “The same baseline was introduced for neural models on CNN/DM by Nallapati et al. (2017) who similarly find that it is extremely effective.”
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, background): “Most prior work on neural abstractive summarization relied on large-scale, high-quality datasets of supervised document-summary pairs (Hermann et al., 2015) and achieved promising results (Rush et al., 2015; Nallapati et al., 2016; See et al., 2017).”
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, background): “The idea of capturing document-level semantic information has been previously explored for recurrent neural networks (Mikolov and Zweig, 2012; Ghosh et al., 2016; Dieng et al., 2017), however, we are not aware of any existing convolutional models.”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, methodology): “…contains news documents and their corresponding highlights crawled from CNN and Daily Mail website, and it is commonly used in extractive summarization (Cheng and Lap-ata 2016; Nallapati, Zhai, and Zhou 2017) and abstractive summarization (Nallapati et al. 2016; See, Liu, and Man-ning 2017).”
Bottom-Up Abstractive Summarization (2018, methodology): “Inspired by Nallapati et al. (2017), we investigate whether aligning the summary and the source during training and ﬁxing the gold copy attention to pick the ”correct” source word is beneﬁ-cial.”
Controllable Abstractive Summarization (2017, methodology): “Research in abstractive summarization with sequence-to-sequence models focuses on neural architectures (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; See et al., 2017; Paulus et al., 2017) and learning objectives (Paulus et al., 2017).”
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “2 The dataset has been used for summarization as is (See et al., 2017), or after pre-processing for entity anonymization (Nallapati et al., 2017).”
Text Summarization with Pretrained Encoders (2019, background): “Rush et al. (2015) and Nallapati et al. (2016) were among the first to apply the neural encoderdecoder architecture to text summarization.”
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents (2018, methodology): “Examples of prominent approaches include frequency-based methods (Vanderwende et al., 2007), graph-based methods (Erkan and Radev, 2004), topic modeling (Steinberger and Jezek, 2004), and neural models (Nallapati et al., 2017).”
Evaluating the Factual Consistency of Abstractive Text Summarization (2019, background): “Common approaches to summarization are extractive (Dorr et al., 2003; Nallapati et al., 2017) where the model directly copies the salient parts of the source document into the summary, abstrac-tive (Rush et al., 2015; Paulus et al., 2017) where the important parts are paraphrased to form novel…”
Neural Text Summarization: A Critical Evaluation (2019, background): “Two strategies of generating summaries are extractive (Dorr et al., 2003; Nallapati et al., 2017), where salient fragments of the source document are identiﬁed and directly copied into the summary, and abstractive (Rush et al., 2015; See et al., 2017), where the salient parts are detected and…”
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, methodology): “One of the most dominant architectures is to employ RNN-based seq2seq models with attention mechanism such as (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; Cohan et al., 2018; Hsu et al., 2018; Gehrmann et al., 2018).”
Deep Communicating Agents for Abstractive Summarization (2018, methodology): “A.1 Datasets CNN/DailyMail: CNN/DailyMail dataset (Nal-lapati et al., 2017; Hermann et al., 2015) is a collection of online news articles along with multi-sentence summaries.”
Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018, methodology): “In our experiments MIXER performed worse than the model of Nallapati et al. (2017) just trained on collective labels.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, methodology): “For all the datasets, we evaluate standard ROUGE1, ROUGE-2, and ROUGE-L (Lin, 2004) on fulllength F1 (with stemming) following previous works (Nallapati et al., 2017; See et al., 2017; Paulus et al., 2018).”
Neural Document Summarization by Jointly Learning to Score and Select Sentences (2018, methodology): “Nallapati et al. (2017) propose a system called SummaRuNNer with more features, which also treat extractive document summarization as a sequence labeling task.”
BanditSum: Extractive Summarization as a Contextual Bandit (2018, methodology): “Recurrent neural networks may be trained with stochastic gradient ascent to maximize the likelihood of a set of ground-truth binary label sequences (Cheng and Lapata, 2016; Nallapati et al., 2017).”