Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

By Greg Durrett, Taylor Berg-Kirkpatrick, Daniel Klein. 2016

171 citations recorded by the source corpus.

We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints. Our model selects textual units to include in the summary based on a rich set of sparse features whose weights are learned on a large corpus. We allow for the deletion of content within a sentence when that deletion is licensed by compression rules; in our framework, these are implemented as dependencies between subsentential units of text. Anaphoricity constraints then improve cross-sentence coherence by guaranteeing that, for each pronoun included in the summary, the pronoun's antecedent is included as well or the pronoun is rewritten as a full mention. When trained end-to-end, our final system outperforms prior work on both ROUGE as well as on human judgments of linguistic quality.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

A Deep Reinforced Model for Abstractive Summarization (2017, methodology): “Even though this dataset has been used to train extractive summarization systems (Durrett et al., 2016; Hong & Nenkova, 2014; Li et al., 2016) or closely-related models for predicting the importance of a phrase in an article (Yang & Nenkova, 2014; Nye & Nenkova, 2015; Hong et al.”
Fine-tune BERT for Extractive Summarization (2019, methodology): “We ﬁrst split sentences by CoreNLP and pre-process the dataset following methods in Durrett et al. (2016).”
MLSUM: The Multilingual Summarization Corpus (2020, methodology): “Although (Grusky et al., 2018) found indications of bias towards extractive approaches, several research efforts have used this dataset for summarization (Hong and Nenkova, 2014; Durrett et al., 2016; Paulus et al., 2017).”
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (2018, methodology): “Following previous work (Durrett et al., 2016; Paulus et al., 2018), we obtained LEAD summaries based on the first 100 words for NY Times documents.”
Bottom-Up Abstractive Summarization (2018, background): “Recent systems such as Durrett et al. (2016) also learn a model to select sentences and then compress them.”
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “Despite this, limited work has used this dataset for summarization (Hong and Nenkova, 2014; Durrett et al., 2016; Paulus et al., 2017).”
Text Summarization with Pretrained Encoders (2019, methodology): “Following Durrett et al. (2016), we split these into 100,834/9,706 training/test examples, based on the date of publication (the test set contains all articles published from January 1, 2007 onward).”
Deep Communicating Agents for Abstractive Summarization (2018, methodology): “New York Times (NYT): Although this dataset has mainly been used to train extractive summarization systems (Hong and Nenkova, 2014; Hong et al., 2015; Li et al., 2016; Durrett et al., 2016), it has recently been used for the abstractive summarization task (Paulus et al., 2018).”
WikiHow: A Large Scale Text Summarization Dataset (2018, methodology): “While this dataset has been mainly used for extractive systems (Hong and Nenkova, 2014; Durrett et al., 2016), Paulus et al. (2017) are the ﬁrst to evaluate their abstractive system using NYT.”
Improving Abstraction in Text Summarization (2018, methodology): “The New York Times dataset (Sand-haus, 2008) has also been used as a benchmark for the generation of long summaries (Durrett et al., 2016; Paulus et al., 2017).”
Content Selection in Deep Learning Models of Summarization (2018, methodology): “We choose to optimize for ROUGE-1 rather than ROUGE-2 similarly to other optimization based approaches to summarization (Sipos et al., 2012; Durrett et al., 2016) which found this to be the easier target to learn.”
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization (2019, methodology): “We conducted our summarization experiments on the non-anonymous version CNN/Dailymail (CNNDM) dataset (Hermann et al., 2015; See et al., 2017), and the New York Times dataset (Durrett et al., 2016; Xu and Durrett, 2019).”
Recent advances in document summarization (2017, methodology): “A very recent system [45] tries to combine discourselevel compression based on RST tree and syntactic compression based on constituent trees.”
Text Generation with Exemplar-based Adaptive Decoding (2019, unknown): “This is a smaller portion of the original test data, after ﬁltering out instances with summaries shorter than 50 tokens ( § 4.2; Durrett et al., 2016).”
Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020, methodology): “Note that we use limited-length ROUGE recall as Durrett et al. (2016), where the selected sentences are truncated to the length of the human-written summaries and the recall scores are used instead of F1.”
Controlling the Amount of Verbatim Copying in Abstractive Summarization (2019, background): “The sentences can be optionally compressed to remove inessential phrases, leading to compressive summaries (Martins and Smith 2009; Li et al. 2013; Wang et al. 2013; Filippova et al. 2015; Durrett, Berg-Kirkpatrick, and Klein 2016).”
Robustness in Coreference Resolution (2020, background): “, 2017), text summarization (Durrett et al., 2016), slot-filling (Yu & Ji, 2016), math problem solving (Matsuzaki et al.”
Annotating and Modeling Fine-grained Factuality in Summarization (2021, background): “, 1999; Carenini and Cheung, 2008) and compression (Knight and Marcu, 2000; Berg-Kirkpatrick et al., 2011; Woodsend and Lapata, 2012; Durrett et al., 2016) in summarization has typically focused evaluation on content selection and grammaticality, with little heed paid to factuality.”
Discourse-Aware Neural Extractive Text Summarization (2019, methodology): “Durrett et al. (2016) presented a system of sentence extraction and compression with ILP methods using discourse structure.”