Neural Summarization by Extracting Sentences and Words

By Jianpeng Cheng, Mirella Lapata. 2016

860 citations recorded by the source corpus.

Traditional approaches to extractive summarization rely heavily on humanengineered features. In this work we propose a data-driven approach based on neural networks and continuous sentence features. We develop a general framework for single-document summarization composed of a hierarchical document encoder and an attention-based extractor. This architecture allows us to develop different classes of summarization models which can extract sentences or words. We train our models on large scale corpora containing hundreds of thousands of document-summary pairs 1 . Experimental results on two summarization datasets demonstrate that our models obtain results comparable to the state of the art without any access to linguistic annotation.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Fine-tune BERT for Extractive Summarization (2019, methodology): “Although many neural models have been proposed for extractive summarization recently (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018; Dong et al., 2018; Zhang et al., 2018; Zhou et al., 2018), the improvement on automatic metrics like ROUGE has reached a bottleneck due to the…”
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond (2016, neutral): “O u t pu t L a y e r”
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, background): “sequently concatenating) the most important sentences in a document (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018b).”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “Hierarchical models which treats sentence selection as a latent variable have been applied text categorization (Yang et al., 2016b), extractive summarization (Cheng and Lapata, 2016), machine translation (Ba et al., 2014) and sentiment analysis (Yessenalina et al., 2010; Lei et al., 2016).”
Denoising Distantly Supervised Open-Domain Question Answering (2018, methodology): “Question answering is one of the most important tasks in NLP.”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, methodology): “(Cheng and Lapata 2016) use a convolutional neural network to encode sentences, and then an RNN reads the sentence representations sequentially to encode the document.”
Bottom-Up Abstractive Summarization (2018, neutral): “In contrast, recent work in neural network based data-driven extractive summarization has focused on extracting and ordering full sentences (Cheng and Lapata, 2016; Dlikman and Last, 2016).”
Incorporating Copying Mechanism in Sequence-to-Sequence Learning (2016, background): “Cheng and Lapata (2016) devised a neural summarization model with the ability to extract words/sentences from the source.”
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “DUC summaries are often used in conjunction with larger training datasets, including Gi-gaword (Rush et al., 2015; Chopra et al., 2016), CNN / Daily Mail (Nallapati et al., 2017; Paulus et al., 2017; See et al., 2017), or Daily Mail alone (Nallapati et al., 2016b; Cheng and Lapata, 2016).”
SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents (2016, methodology): “In addition, we also compare with the state-of-the art deep learning models from (Cheng and Lapata 2016) and (Nallapati et al. 2016).”
Beyond Goldfish Memory: Long-Term Open-Domain Conversation (2021, background): “Summarization is a rich ﬁeld where the vast majority of work focuses on summarizing documents (Kaikhah, 2004; Kry´s-1 ci´nski et al., 2019; Cheng and Lapata, 2016), for example summarizing in order to predict other relevant information (West et al., 2019), and there is some work on dialogue as well…”
Deep Communicating Agents for Abstractive Summarization (2018, background): “…for summarization as well (Rush et al., 2015; Zeng et al., 2016; Nallapati et al., 2017), and pointer networks (Vinyals et al., 2015a), in particular, help address redundancy and saliency in generated summaries (Cheng and Lapata, 2016; See et al., 2017; Paulus et al., 2018; Fan et al., 2017).”
Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018, methodology): “Recent deep learning methods (Kågebäck et al., 2014; Yin and Pei, 2015; Cheng and Lapata, 2016; Nallapati et al., 2017) learn continuous features without any linguistic preprocessing (e.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, methodology): “For neural models, Cheng and Lapata (2016) used a second neural net to select words from an ex-tractor’s output.”
R3: Reinforced Reader-Ranker for Open-Domain Question Answering (2017, neutral): “(Cheng and Lapata, 2016) and (Choi et al., 2017) proposed coarse-to-ﬁne frameworks with an additional sentence-level prediction followed by the original word-level prediction for text summarization and reading comprehension, respectively.”
This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation (2019, positive): “Suppose our input is an email body D which consists of | D | sentences: We ﬁrst use a temporal CNN (Kim, 2014) to build individual sentence representations.”
Neural Document Summarization by Jointly Learning to Score and Select Sentences (2018, methodology): “(Cheng and Lapata, 2016), N EU S UM performs signiﬁcantly better in terms of R OUGE -1, R OUGE - 2 and R OUGE -”
BanditSum: Extractive Summarization as a Contextual Bandit (2018, methodology): “Recurrent neural networks may be trained with stochastic gradient ascent to maximize the likelihood of a set of ground-truth binary label sequences (Cheng and Lapata, 2016; Nallapati et al., 2017).”