Multi-Reward Reinforced Summarization with Saliency and Entailment

By Ramakanth Pasunuru, Mohit Bansal. 2018

211 citations recorded by the source corpus.

Abstractive text summarization is the task of compressing and rewriting a long document into a short summary while maintaining saliency, directed logical entailment, and non-redundancy. In this work, we address these three important aspects of a good summary via a reinforcement learning approach with two novel reward functions: ROUGESal and Entail, on top of a coverage-based baseline. The ROUGESal reward modifies the ROUGE metric by up-weighting the salient phrases/words detected via a keyphrase classifier. The Entail reward gives high (length-normalized) scores to logically-entailed summaries using an entailment classifier. Further, we show superior performance improvement when these rewards are combined with traditional metric (ROUGE) based rewards, via our novel and effective multi-reward approach of optimizing multiple rewards simultaneously in alternate mini-batches. Our method achieves the new state-of-the-art results on CNN/Daily Mail dataset as well as strong improvements in a test-only transfer setup on DUC-2002.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Reinforcement learning has pushed performance using common summarization metrics [Paulus et al., 2017] as well as alternative metrics that transfer knowledge from another task [Pasunuru et al., 2017, Pasunuru and Bansal, 2018].”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “For example, Paulus et al. (2018) demonstrated that we may be able to optimize ROUGE directly and Pasunuru and Bansal (2018) explored alternative optimization targets such as maximizing entailment.”
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, background): “Abstractive systems trained on these datasets often have a hard time beating the LEAD, let alone EXTORACLE, or display a low degree of novelty in their summaries (See et al., 2017; Tan and Wan, 2017; Paulus et al., 2018; Pasunuru and Bansal, 2018; Celikyilmaz et al., 2018).”
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “Furthermore, we also apply the multi-reward optimization strategy (Pasunuru and Bansal, 2018) to train the model with two mixed losses alternately with an alternate rate n : m , i.e., train with L for n mini-batches, then train with L for m mini-batches, repeat until convergence. n and m are two…”
Survey of Hallucination in Natural Language Generation (2022, neutral): “Some research [3, 73, 84, 102, 120, 135, 163] adopt reinforcement learning to solve the hallucination problem, which utilizes different rewards to optimize the model.”
Bottom-Up Abstractive Summarization (2018, neutral): “90 Saliency + Entailment reward (Pasunuru and Bansal, 2018) 40.”
Evaluating the Factual Consistency of Abstractive Text Summarization (2019, background): “, 2018) and multi-reward (Pasunuru and Bansal, 2018) settings where the additional task and reward was textual entailment (NLI).”
Neural Text Summarization: A Critical Evaluation (2019, methodology): “…and copying mechanisms (See et al., 2017; Tan et al., 2017; Cohan et al., 2018), multi-task and multi-reward training techniques (Guo et al., 2018; Pasunuru and Bansal, 2018; Kry´sci´nski et al., 2018), reinforcement learning strategies (Paulus et al., 2017; Narayan et al., 2018b; Dong et al.,…”
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, background): “, 2018), reinforcement learning (Paulus et al., 2018; Pasunuru and Bansal, 2018), contextual agent attention (Celikyilmaz et al.”
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models (2019, methodology): “Recent research works (Paulus et al., 2017; Pasunuru and Bansal, 2018; Arumae and Liu, 2019) have proposed to use evaluation metrics – and ROUGE in particular – to learn the model parameters through Reinforcement Learning (RL) techniques.”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, background): “Recent works have explored applying RL on tasks such as question-answering [84, 85, 47, 83, 84], summarization [58, 53, 70, 60, 17, 51], and machine translation [58, 87, 79, 82, 81, 13, 66, 5, 49].”
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference (2019, methodology): “Moreover, Guo et al. (2018) and Pasunuru and Bansal (2018) proposed to use NLI-based loss functions or multi-task learning with NLI for document summarization.”
Improving Abstraction in Text Summarization (2018, background): “entailment (Pasunuru and Bansal, 2018) with reinforcement learning; as well as generative adversarial networks to generate more natural summaries (Liu et al.”
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer (2019, background): “extensive research on model augmentations for text summarization (Gülcehre et al., 2016; Pasunuru and Bansal, 2018; Li et al., 2018), they are often complicated or summarization-specific.”
Planning with Learned Entity Prompts for Abstractive Summarization (2021, neutral): “(2020) and report on textual entailment (Pasunuru and Bansal, 2018; Falke et al., 2019; Kryscinski et al., 2020).”
An Overview of Natural Language State Representation for Reinforcement Learning (2020, background): “In most cases, the state representation was, as in other tasks, the hidden state of an RNN generating the summary (Ling & Rush, 2017; Pasunuru & Bansal, 2018), also together with an encoding of a candidate sentence (Lee & Lee, 2017).”