Extractive Summarization of Long Documents by Combining Global and Local Context

By Wen Xiao, Giuseppe Carenini. 2019

166 citations recorded by the source corpus.

In this paper, we propose a novel neural single-document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic. We evaluate the model on two datasets of scientific papers , Pubmed and arXiv, where it outperforms previous work, both extractive and abstractive models, on ROUGE-1, ROUGE-2 and METEOR scores. We also show that, consistently with our goal, the benefits of our method become stronger as we apply it to longer documents. Rather surprisingly, an ablation study indicates that the benefits of our model seem to come exclusively from modeling the local context, even for the longest documents.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Recursively Summarizing Books with Human Feedback (2021, background): “…kinds of long documents, such as scientiﬁc papers (Abu-Jbara and Radev, 2011; Collins et al., 2017; Subramanian et al., 2019; Cohan et al., 2018; Xiao and Carenini, 2019; Zhao et al., 2020; Sotudeh et al., 2020), and patents (Sharma et al., 2019), as well as multi-document summarization (Liu…”
On Generating Extended Summaries of Long Documents (2020, background): “More recently, Xiao and Carenini (2019) proposed a sequence-to-sequence model that incorporates both the global context of the entire document and local context within the specified section.”
Efficient Attentions for Long Document Summarization (2021, methodology): “Popular methods rely on extractive summarizers that identify salient sentences based on positional information (Dong et al., 2020) or combined global and local contexts (Xiao and Carenini, 2019), where each sentence is represented as aggregated word embeddings.”
TLDR: Extreme Summarization of Scientific Documents (2020, background): “, 2018), and other techniques focusing on exploiting the unique properties of scientific documents such as long length and structure (Conroy and Davis, 2017; Nikolov et al., 2018; Cohan et al., 2018; Xiao and Carenini, 2019).”
GUIR @ LongSumm 2020: Learning to Generate Long Summaries from Scientific Documents (2020, background): “More recently, Xiao and Carenini (2019) proposed a sequence-to-sequence model which incorporates both the global context of the entire document, and local context within the speciﬁed section.”
Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization (2021, methodology): “Inspired by prior works (Yu et al., 2019; Tsai et al., 2019), we propose a vision guided multi-head attention mechanism for the text-vision fusion.”