Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning

By Cody Rioux, Sadid A. Hasan, Yllias Chali. 2014

41 citations recorded by the source corpus.

This paper explores alternate algorithms, reward functions and feature sets for performing multi-document summarization using reinforcement learning with a high focus on reproducibility. We show that ROUGE results can be improved using a unigram and bigram similarity metric when training a learner to select sentences for summarization. Learners are trained to summarize document clusters based on various algorithms and reward functions and then evaluated using ROUGE. Our experiments show a statistically significant improvement of 1.33%, 1.58%, and 2.25% for ROUGE-1, ROUGE-2 and ROUGEL scores, respectively, when compared with the performance of the state of the art in automatic summarization with reinforcement learning on the DUC2004 dataset. Furthermore query focused extensions of our approach show an improvement of 1.37% and 2.31% for ROUGE-2 and ROUGE-SU4 respectively over query focused extensions of the state of the art with reinforcement learning on the DUC2006 dataset.

Subfields: Artificial Intelligence, Natural Language Processing, Summarization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Summarization

Citations in the corpus

Better Rewards Yield Better Summaries: Learning to Summarise Without References (2019, methodology): “As an alternative, input-speciﬁc RL (Rioux et al., 2014; Ryang and Abekawa, 2012) does not require reference summaries: for each input document at test time, a summarisation policy is trained specifically for the input, by letting the RL summariser interact with a heuristic-based reward function,…”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, background): “Although (Ryang and Abekawa 2012; Rioux, Hasan, and Chali 2014; Henb, Mieskes, and Gurevych 2015) use value-based RL algorithms for extractive summarization, all of them are based on handcrafted features and do not consider coherence as the part of the reward.”
BanditSum: Extractive Summarization as a Contextual Bandit (2018, methodology): “The use of reinforcement learning (RL) in extractive summarization was ﬁrst explored by Ryang and Abekawa (2012), who proposed to use the TD( λ ) algorithm to learn a value function for sentence selection.”
Preference-based interactive multi-document summarisation (2019, methodology): “Most existing RL-based document sum-marisation systems either use heuristic functions (e.g., Ryang and Abekawa 2012; Rioux et al. 2014), which do not rely on reference summaries, or ROUGE scores requiring reference summaries as the rewards for RL (Paulus et al. 2017; Pasunuru and Bansal 2018;…”
A Reinforcement Learning Approach for Adaptive Single- and Multi-Document Summarization (2015, methodology): “Rioux et al. (2014) achieved a higher R-2 score.”
APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning (2018, methodology): “The learn-Method ing parameters we use for TD( λ ) are the same as those by Rioux et al. (2014).”
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization (2020, methodology): “For baselines, we consider TF-IDF, which computes the cosine similarity of the tf-idf vectors of source and summaries; JS, which computes the JS divergence between the words distributions in source documents and summaries; and the REAPER heuristics proposed by Rioux et al. (2014).”
Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation (2019, methodology): “We formulate the optimisation problems M x for NLG tasks as episodic MDP [ Ryang and Abekawa, 2012; Rioux et al. , 2014 ] .”
Recent advances in document summarization (2017, background): “4) System ensemble [74,152,186], indirect supervision [87,162], neural network rankers [16,124,206]”
An Overview of Natural Language State Representation for Reinforcement Learning (2020, methodology): “Initial works applying RL used tuples of features to represent the state, composed of the summary at each time step, a history of actions and a binary variable indicating the terminal state (Ryang & Abekawa, 2012; Rioux et al., 2014; Henß et al., 2015).”