Ranking Sentences for Extractive Summarization with Reinforcement Learning

By Shay B. Cohen, Mirella Lapata. 2018

587 citations recorded by the source corpus.

Single document summarization is the task of producing a shorter version of a document while preserving its principal information content. In this paper we conceptualize extractive summarization as a sentence ranking task and propose a novel training algorithm which globally optimizes the ROUGE evaluation metric through a reinforcement learning objective. We use our algorithm to train a neural summarization model on the CNN and DailyMail datasets and demonstrate experimentally that it outperforms state-of-the-art extractive and abstractive systems when evaluated automatically and by humans.

Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Fine-tune BERT for Extractive Summarization (2019, methodology): “• R EFRESH (Narayan et al., 2018) is an extractive summarization system trained by globally optimizing the ROUGE metric with reinforcement learning.”
MLSUM: The Multilingual Summarization Corpus (2020, methodology): “A similar approach has been adopted for several dataset releases in the recent past, such as Question Answering Corpus (Hermann et al., 2015b) or XSUM (Narayan et al., 2018a).”
Better Rewards Yield Better Summaries: Learning to Summarise Without References (2019, methodology): “5 (Narayan et al., 2018b) R-1,2,L 40.”
On Faithfulness and Factuality in Abstractive Summarization (2020, methodology): “[66.7, 40.0, 51.9] Figure 1: Hallucinations in extreme document summarization: the abbreviated article, its gold summary and the abstractive model generated summaries (P T G EN , See et al. 2017; TC ONV S2S, Narayan et al. 2018a; and, GPT-T UNED , T RAN S2S and B ERT S2S, Rothe et al. 2020) for a ne”
Text Summarization with Pretrained Encoders (2019, methodology): “We report experiments following a question-answering (QA) paradigm (Clarke and Lapata, 2010; Narayan et al., 2018b) which quantiﬁes the degree to which summarization models retain key information from the document.”
Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model (2019, background): “…output summaries from document clusters on the same topic, has largely been performed on datasets with less than 100 document clusters such as the DUC 2004 (Paul and James, 2004) and TAC 2011 (Owczarzak and Dang, 2011) datasets, and has beneﬁted less from advances in deep learning methods.”
Neural Text Summarization: A Critical Evaluation (2019, methodology): “…techniques (Guo et al., 2018; Pasunuru and Bansal, 2018; Kry´sci´nski et al., 2018), reinforcement learning strategies (Paulus et al., 2017; Narayan et al., 2018b; Dong et al., 2018; Wu and Hu, 2018), and hybrid extractive-abstractive models (Liu et al., 2018; Hsu et al., 2018; Gehrmann et…”
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, methodology): “Our experimental results show that the proposed MMN model improves abstractive summarization performance on both our new Reddit TIFU and existing Newsroom-Abs (Grusky et al., 2018) and XSum (Narayan et al., 2018a) datasets.”
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks (2019, methodology): “We evaluate our setups on three different summarization datasets of varying characteristics: Gigaword (Napoles et al., 2012), CNN and DailyMail (Hermann et al., 2015), and BBC extreme (Narayan et al., 2018).”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “See et al. (2017) 37.22 15.78 33.90 rnn-ext + abs + RL 39.46 17.34 36.72 Table 3: Generalization to DUC-2002 (F1). current sentence-ranking RL model by Narayan et al. (2018), showing that our pointer-network extractor and reward formulations are very effective when combined with A2C RL.”
This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation (2019, background): “Past NLP email research has focused on summarization (Muresan et al., 2001; Nenkova and Bagga, 2003; Rambow et al., 2004; Corston-Oliver et al., 2004; Wan and McKeown, 2004; Carenini et al., 2007; Zajic et al., 2008; Carenini et al., 2008; Ulrich et al., 2009), keyword extraction and action…”
BanditSum: Extractive Summarization as a Contextual Bandit (2018, methodology): “Narayan et al. (2018) uses a cascade of ﬁlters in order to arrive at a set of candidate extractive summaries, which we can regard as an approximation of the true action space.”
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models (2019, methodology): “Narayan et al. (2018) used such an approach in an extractive summarization setting, learning to select the most relevant sentences within the input text in order to construct its summary.”
Question Answering as an Automatic Evaluation Metric for News Article Summarization (2019, methodology): “In extractive summarization, summaries are created by selecting a collection of key sentences from the source document ( e.g. , Nallapati et al. (2017); Narayan et al. (2018)).”
Evaluation of Text Generation: A Survey (2020, background): “Fluency is also adopted in several text generation tasks including document summarization (Celikyilmaz et al., 2018; Narayan et al., 2018), recipe generation (Bosselut et al., 2018), image captioning (Lan et al., 2017), video description generation (Park et al., 2018), and question generation (Du…”
BARTScore: Evaluating Generated Text as Text Generation (2021, neutral): “Extractive Systems E1: BanditSum [11]; E2: Refresh [49]; E3: NeuSum [81]; E4: LSTM-PN-RL [80]; E5: BERT-TF-SL [80]; E6: BERT-TF-PN [80]; E7: BERT-LSTM-PN-RL [80]; E8: BERT-LSTM-PN [80]; E9: HeterGraph [69]; E10: MatchSum [79].”
Guiding Extractive Summarization with Question-Answering Rewards (2019, methodology): “These studies build distributed sentence representations using neural networks (Cheng and Lapata, 2016; Yasunaga et al., 2017) and use reinforcement learning to optimize the evaluation metric (Narayan et al., 2018b) and improve summary coherence (Wu and Hu, 2018).”
CTRLsum: Towards Generic Controllable Text Summarization (2020, background): “…systems: extractive summarization where models find and copy important portions of a documents (Cheng and La-pata, 2016; Nallapati et al., 2017; Narayan et al., 2018), and abstractive summarization where models freely generates novel sentences (Rush et al., 2015; See et al., 2017; Paulus et…”
Neural Latent Extractive Document Summarization (2018, methodology): “…(Nallapati et al. 2017; SummaRuNNer ), the model described in Section 2.1 ( EXTRACT ) which encodes sentences using LSTMs, a variant which employs CNNs instead ( Cheng and Lapata 2016; EXTRACT - CNN ), as well as a similar system based on reinforcement learning ( Narayan et al. 2018; R EFRESH ).”