Abstractive Document Summarization with a Graph-Based Attentional Neural Model
By Jiwei Tan, Xiaojun Wan, Jianguo Xiao. 2017
303 citations recorded by the source corpus.
Abstractive summarization is the ultimate goal of document summarization research, but previously it is less investigated due to the immaturity of text generation techniques. Recently impressive progress has been made to abstractive sentence summarization using neural models. Unfortunately, attempts on abstractive document summarization are still in a primitive stage, and the evaluation results are worse than extractive methods on benchmark datasets. In this paper, we review the difficulties of neural abstractive document summarization, and propose a novel graph-based attention mechanism in the sequence-to-sequence framework. The intuition is to address the saliency factor of summarization, which has been overlooked by prior works. Experimental results demonstrate our model is able to achieve considerable improvement over previous neural abstractive models. The data-driven neural abstractive method is also competitive with state-of-the-art extractive methods.
Subfields: Artificial Intelligence, Natural Language Processing, Summarization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Caiming Xiong
- Jianfeng Gao
- Yejin Choi
- Richard Socher
- Percy Liang
- Graham Neubig
- Luke S. Zettlemoyer
In Natural Language Processing
- Mohit Bansal
- Caiming Xiong
- Yejin Choi
- Mirella Lapata
- Richard Socher
- Jianfeng Gao
- Graham Neubig
- Luke S. Zettlemoyer
In Summarization
- Andreas Vlachos
- Hardy Hardy
- Bryan McCann
- Richard Socher
- Wojciech Kryscinski
- Yejin Choi
- Max Grusky
- Ramakanth Pasunuru
Citations in the corpus
- Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, background): “extractive ones or are mostly extractive, exhibiting a small degree of abstraction (See et al., 2017; Tan and Wan, 2017; Paulus et al., 2018; Pasunuru and Bansal, 2018; Celikyilmaz et al., 2018).”
- Bottom-Up Abstractive Summarization (2018, background): “Tan et al. (2017) describe a graphbased attention to attend to one sentence at a time, Chen and Bansal (2018) first extract full sentences from a document and then compress them, and Hsu et al. (2018) modulate the attention based on how likely a sentence is included in a summary.”
- Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “We select two semantic and two syntactic dimensions for evaluation based on experiments with evaluation tasks by Paulus et al. (2017) and Tan et al. (2017).”
- Neural Text Summarization: A Critical Evaluation (2019, methodology): “Current approaches to text summarization utilize advanced attention and copying mechanisms (See et al., 2017; Tan et al., 2017; Cohan et al., 2018), multi-task and multi-reward training techniques (Guo et al., 2018; Pasunuru and Bansal, 2018; Kry´sci´nski et al., 2018), reinforcement learning…”
- Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, background): “Some notable examples include the use of variational autoencoders (VAEs) (Miao and Blunsom, 2016; Li et al., 2017), graph-based attention (Tan et al., 2017), pointer-generator models (See et al., 2017), self-attention networks (Liu et al., 2018), reinforcement learning (Paulus et al., 2018;…”
- Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018, methodology): “Extractive vs Abstractive Systems Our automatic evaluation results further demonstrate that REFRESH is superior to abstractive systems (Chen et al., 2016; Nallapati et al., 2016; See et al., 2017; Tan and Wan, 2017) which are all variants of an encoder-decoder architecture (Sutskever et al.”
- BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization (2019, background): “Recent advances in abstractive summarization show promising results in generating fluent and informative summaries (Rush et al., 2015; Nalla-pati et al., 2016; Tan et al., 2017; Paulus et al., 2017).”
- Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “…coverage (Suzuki and Nagata, 2016; Chen et al., 2016; See et al., 2017), RL based metric optimization (Paulus et al., 2018), graph-based attention (Tan et al., 2017), and the copy mechanism (Miao and Blunsom, 2016; Gu et al., 2016; See et al., 2017). important information been generated); while…”
- Guiding Extractive Summarization with Question-Answering Rewards (2019, background): “Graph attention (Tan et al., 2017) introduces a graph-based attention mechanism to enhance the encoderdecoder framework.”
- Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation (2018, methodology): “Recent work has also adopted machine translation inspired neural seq2seq models for abstractive summarization with advances in hierarchical, distractive, saliency, and graph-attention modeling (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; Chen et al., 2016; Tan et al., 2017).”
- Entity Commonsense Representation for Neural Abstractive Summarization (2018, methodology): “We note that our proposed module can be easily attached to more sophisticated abstractive summarization models (Zhou et al., 2017; Tan et al., 2017) that are based on the traditional encoderdecoder framework and consequently can produce better results.”
- Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020, background): “…source sequence were initially proposed for machine translation (MT) (Sutskever et al., 2014; Kalchbrenner and Blunsom, 2013; Bahdanau et al., 2015; Vaswani et al., 2017), but are now used widely as a central component of a variety of NLP systems (e.g. Tan et al. (2017); Chollampatt and Ng (2018)).”
- Efficient Adaptation of Pretrained Transformers for Abstractive Summarization (2019, background): “Attention mechanisms have been shown to improve a variety of models [14; 25; 3], and is one of the motivating factors for this work.”
- An Introductory Survey on Attention Mechanisms in NLP Problems (2018, background): “Tan et al. proposed an alternative for computing attention scores for the task of abstractive document summarization (Tan, Wan, and Xiao 2017).”
- Natural Language Processing Advancements By Deep Learning: A Survey (2020, background): “However, since simple attention models perform worse than extractive models, therefore more effective attention models such as graph-based attention [155] and transformers [137] have been proposed for this task.”
- Joint Parsing and Generation for Abstractive Summarization (2019, methodology): “At decoding time, we apply beam search with reference (Tan, Wan, and Xiao 2017) to generate summary sequences.”
- DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks (2019, background): “…have led to great progress in important down-stream NLP tasks like text summarization (Rush et al., 2015; Nallapati et al., 2016; See et al., 2017; Tan et al., 2017; Yavuz et al., 2018), machine translation (Cho et al., 2014; Sutskever et al., 2014; Luong et al., 2015; Bahdanau et al., 2015),…”