Deep Communicating Agents for Abstractive Summarization

By Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi. 2018

317 citations recorded by the source corpus.

We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization. With deep communicating agents, the task of encoding a long text is divided across multiple collaborating agents, each in charge of a subsection of the input text. These encoders are connected to a single decoder, trained end-to-end using reinforcement learning to generate a focused and coherent summary. Empirical results demonstrate that multiple communicating encoders lead to a higher quality summary compared to several strong baselines, including those based on a single encoder or multiple non-communicating encoders.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Fine-tune BERT for Extractive Summarization (2019, background): “• D CA (Celikyilmaz et al., 2018) is the Deep Communicating Agents, a state-of-the-art ab-stractive summarization system with multiple agents to represent the document as well as hierarchical attention mechanism over the agents for decoding.”
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, background): “…has consistently attracted attention (Cheng and Lapata, 2016; Durrett et al., 2016; Nallapati et al., 2016, 2017; See et al., 2017; Tan and Wan, 2017; Narayan et al., 2017; Fan et al., 2017; Paulus et al., 2018; Pasunuru and Bansal, 2018; Celikyilmaz et al., 2018; Narayan et al., 2018a,b).”
ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, methodology): “RL for text generation Since many metrics of interest in NLP are non-differentiable, several approaches used RL for text generation [8, 34, 30, 6].”
Pay Less Attention with Lightweight and Dynamic Convolutions (2019, neutral): “We also show results for a reinforcement learning approach (Celikyilmaz et al., 2018) and note that RL is equally applicable to our architecture.”
Depth-Adaptive Transformer (2019, neutral): “, 2019), summarization (Celikyilmaz et al., 2018; Wu et al., 2019) and natural language understanding (Radford et al.”
Bottom-Up Abstractive Summarization (2018, background): “Here, the model outperforms the RL based model by Paulus et al. (2017) in ROUGE-1 and 2, but not L, and is comparable to the results of (Celikyilmaz et al., 2018) except for ROUGE-L.”
Text Summarization with Pretrained Encoders (2019, background): “Celikyilmaz et al. (2018) propose an abstractive system where multiple agents (encoders) represent the document together with a hierarchical attention mechanism (over the agents) for decoding.”
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, background): “…pointer-generator models (See et al., 2017), self-attention networks (Liu et al., 2018), reinforcement learning (Paulus et al., 2018; Pasunuru and Bansal, 2018), contextual agent attention (Celikyilmaz et al., 2018) and integration with extractive models (Hsu et al., 2018; Gehrmann et al., 2018).”
Learning Attentional Communication for Multi-Agent Cooperation (2018, background): “It is worth mentioning that CommNet has been extended for abstractive summarization [2] in natural language processing.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, methodology): “(2018) use RL for ranking sentences in pure extraction-based summarization and Çelikyilmaz et al. (2018) investigate multiple communicating encoder agents to enhance the copying abstractive summarizer.”
Question Answering as an Automatic Evaluation Metric for News Article Summarization (2019, methodology): “Current models (Nallapati et al., 2016; See et al., 2017; Paulus et al., 2017; Celikyilmaz et al., 2018) made various adjustments to sequence-to-sequence models to gain improvements in ROUGE (Lin, 2004) scores.”
Evaluation of Text Generation: A Survey (2020, background): “Fluency is also adopted in several text generation tasks including document summarization (Celikyilmaz et al., 2018; Narayan et al., 2018), recipe generation (Bosselut et al., 2018), image captioning (Lan et al., 2017), video description generation (Park et al., 2018), and question generation (Du…”
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer (2019, methodology): “Lastly, we provide scores from the state-of-the-art LSTM based model (Celikyilmaz et al., 2018), which includes a reinforcement”
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization (2019, neutral): “Pointer+Coverage (See et al., 2017), Abstract-ML+RL (Paulus et al., 2017) and DCA (Celikyilmaz et al., 2018) are all sequence to sequence learning based models with copy and coverage modeling, reinforcement learning and deep communicating agents extensions.”
Pragmatically Informative Text Generation (2019, result): “…taken from Chen and Bansal (2018); we obtained comparable performance in our reproduction 3 ), an ensemble of two of these base models, and the best previous ab-stractive summarization result for each metric on this dataset (Celikyilmaz et al., 2018; Paulus et al., 2018; Chen and Bansal, 2018).”
Putting the Horse before the Cart: A Generator-Evaluator Framework for Question Generation from Text (2018, unknown): “Very recently deep reinforcement learning has been successfully applied to natural language generation tasks such as abstractive summarization (Paulus et al., 2018; Celikyilmaz et al., 2018) and dialogue generation (Li et al., 2016).”
Automatic Document Sketching: Generating Drafts from Analogous Texts (2021, methodology): “…using a vanilla policy gradient (PG) or a self-critical sequence training (TD-SCST) algorithm (Rennie et al., 2017; Paulus et al., 2018; Pasunuru and Bansal, 2018), can lead to instability during training as the reward cannot be calculated until the end of generation (Celikyilmaz et al., 2018).”