Graph-based Neural Multi-Document Summarization

By Michihiro Yasunaga, Rui Zhang, Kshitijh Meelu, Ayush Pareek, Krishnan Srinivasan, Dragomir R. Radev. 2017

280 citations recorded by the source corpus.

We propose a neural multi-document summarization system that incorporates sentence relation graphs. We employ a Graph Convolutional Network (GCN) on the relation graphs, with sentence embeddings obtained from Recurrent Neural Networks as input node features. Through multiple layer-wise propagation, the GCN generates high-level hidden sentence features for salience estimation. We then use a greedy heuristic to extract salient sentences that avoid redundancy. In our experiments on DUC 2004, we consider three types of sentence relation graphs and demonstrate the advantage of combining sentence relations in graphs with the representation power of deep neural networks. Our model improves upon other traditional graph-based extractive approaches and the vanilla GRU sequence model with no graph, and it achieves competitive results against other state-of-the-art multi-document summarization systems.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback (2020, methodology): “Several works use GNNs to model the structure of text (Yasunaga et al., 2017; Zhang et al., 2018) and more recently, source code (Allamanis et al.”
Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018, background): “A few recent approaches (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2017; Yasunaga et al., 2017) conceptualize extractive summarization as a sequence labeling task in which each label specifies whether each document sentence should be included in the summary.”
BanditSum: Extractive Summarization as a Contextual Bandit (2018, background): “Many extractive summarizers work by selecting sentences from the input document (Luhn, 1958; Mihalcea and Tarau, 2004; Wong et al., 2008; K˚ageb¨ack et al., 2014; Yin and Pei, 2015; Cao et al., 2015; Yasunaga et al., 2017).”
Guiding Extractive Summarization with Question-Answering Rewards (2019, methodology): “These studies build distributed sentence representations using neural networks (Cheng and Lapata, 2016; Yasunaga et al., 2017) and use reinforcement learning to optimize the evaluation metric (Narayan et al., 2018b) and improve summary coherence (Wu and Hu, 2018).”
Content Selection in Deep Learning Models of Summarization (2018, background): “Other recent neural network approaches include, Yasunaga et al. (2017), who learn a graph-convolutional network (GCN) for multi-document summarization.”
LinkBERT: Pretraining Language Models with Document Links (2022, methodology): “For each documentX(i), we use the common TF-IDF cosine similarity metric (Chen et al., 2017; Yasunaga et al., 2017) to obtain top-k documents X(j)’s and make edges (X(i),X(j)).”
Hierarchical Transformers for Multi-Document Summarization (2019, methodology): “Our second graph aims to capture discourse relations (Christensen et al., 2013); it builds an Approximate Discourse Graph (ADG) (Yasunaga et al., 2017) over paragraphs; edges between paragraphs are drawn by counting (a) co-occurring entities and (b) discourse markers (e.g., however, nevertheless)…”
Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020, background): “Recently, some works account for discourse inter-sentential relationships when building summarization graphs, such as the Ap-proximate Discourse Graph (ADG) with sentence personalization features (Yasunaga et al., 2017) and Rhetorical Structure Theory (RST) graph (Xu et al., 2019).”
PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization (2021, background): “The graph-based models typically leverage additional information, e.g. contextual representation (Yasunaga et al., 2017), AMR (Liao et al., 2018), or discourse structure (Christensen et al., 2013; Li et al., 2020), to build a graph for the documents within the cluster that connects information…”
Neural Extractive Summarization with Side Information (2017, background): “Extractive summarization systems select sentences from the document and assemble them together to often generate a grammatical, ﬂuent and semantically correct summary (Cheng and Lapata 2016; Nallapati, Zhai, and Zhou 2017; Yasunaga et al. 2017).”
Multimodal Abstractive Summarization for How2 Videos (2019, background): “…on single-document summarization for domains such as news (Rush et al., 2015; Nallapati et al., 2016; See et al., 2017; Narayan et al., 2018) and some on multi-document summarization (Gold-stein et al., 2000; Lin and Hovy, 2002; Woodsend and Lapata, 2012; Cao et al., 2015; Yasunaga et al., 2017).”
Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs (2019, background): “…Marcu, 2002; Gillick and Favre, 2009; Berg-Kirkpatrick et al., 2011; Di Fabbrizio et al., 2014; Bing et al., 2015; Cao et al., 2017) including maximal marginal relevance (Fabbri et al., 2019), and incorporating queries (Baumel et al., 2018) and graphs (Ganesan et al., 2010; Yasunaga et al., 2017).”
Multi-document Summarization via Deep Learning Techniques: A Survey (2020, methodology): “The sentence relation graph is able to model hierarchical relations among multi-documents as well [5, 141, 142].”
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021, methodology): “Several works use GNNs to model the structure of text (Yasunaga et al., 2017; Zhang et al., 2018; Yasunaga and Liang, 2020) or KGs (Wang et al., 2020a).”