A Divide-and-Conquer Approach to the Summarization of Long Documents
By Alexios Gidiotis, Grigorios Tsoumakas. 2020
136 citations recorded by the source corpus.
We present a novel divide-and-conquer method for the neural summarization of long documents. Our method exploits the discourse structure of the document and uses sentence similarity to split the problem into an ensemble of smaller summarization problems. In particular, we break a long document and its summary into multiple source-target pairs, which are used for training a model that learns to summarize each part of the document separately. These partial summaries are then combined in order to produce a final complete summary. With this approach we can decompose the problem of long document summarization into smaller and simpler problems, reducing computational complexity and creating more training examples, which at the same time contain less noise in the target summaries compared to the standard approach. We demonstrate that this approach paired with different summarization models, including sequence-to-sequence RNNs and Transformers, can lead to improved summarization performance. Our best models achieve results that are on par with the state-of-the-art in two two publicly available datasets of academic articles.
Subfields: Artificial Intelligence, Natural Language Processing, Summarization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ji-rong Wen
- Wayne Xin Zhao
- Jian-Yun Nie
- Heng Ji
- Nikolaus Nova Parulian
- Tianyi Tang
- Junyi Li
- Luyang Robby Huang
In Natural Language Processing
- Jian-Yun Nie
- Heng Ji
- Nikolaus Nova Parulian
- Ji-rong Wen
- Tianyi Tang
- Junyi Li
- Luyang Robby Huang
- Lu Wang
In Summarization
Citations in the corpus
- Sparse is Enough in Scaling Transformers (2021, neutral): “[46, 45, 10] on the ArXiv summarization task, without using the Pegasus loss and without beam search.”
- Efficient Attentions for Long Document Summarization (2021, methodology): “…models: PEGASUS (Zhang et al., 2019), a large pre-trained summarization model with truncated inputs; TLM (Pilault et al., 2020), DANCER (Gidiotis and Tsoumakas, 2020), and SEAL (Zhao et al., 2020), all of them using hybrid extract-then-abstract methods; and B IG B IRD (Za-heer et al.,…”
- Pre-Trained Language Models for Text Generation: A Survey (2022, background): “By splitting the long document into short sentences, it is easier to summarize each short part of the document separately [56], reducing the computational complexity.”
- Deep Literature Survey Automation with an Iterative Workflow (2025, unknown): “Second, the survey structure can be incoherent (Fabbri et al., 2019; Gidiotis & Tsoumakas, 2020; Yang et al., 2023a).”