Overcoming the Lack of Parallel Data in Sentence Compression

By Katja Filippova, Yasemin Altun. 2013

178 citations recorded by the source corpus.

A major challenge in supervised sentence compression is making use of rich feature representations because of very scarce parallel data. We address this problem and present a method to automatically build a compression corpus with hundreds of thousands of instances on which deletion-based algorithms can be trained. In our corpus, the syntactic trees of the compressions are subtrees of their uncompressed counterparts, and hence supervised systems which require a structural alignment between the input and output can be successfully trained. We also extend an existing unsupervised compression method with a learning module. The new system uses structured prediction to learn from lexical, syntactic and other features. An evaluation with human raters shows that the presented data harvesting method indeed produces a parallel corpus of high quality. Also, the supervised system trained on this corpus gets high scores both from human raters and in an automatic evaluation setting, significantly outperforming a strong baseline.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Deep Reinforced Model for Abstractive Summarization (2017, background): “Most summarization models studied in the past are extractive in nature (Neto et al., 2002; Dorr et al., 2003; Filippova and Altun, 2013; Col-menares et al., 2015; Nallapati et al., 2017 A well-studied set of summarization tasks is the Document Understanding Conference (DUC) .”
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond (2016, background): “A vast majority of past work in summarization has been extractive, which consists of identifying key sentences or passages in the source document and reproducing them as summary (Neto et al., 2002; Erkan and Radev, 2004; Wong et al., 2008a; Filippova and Altun, 2013; Colmenares et al., 2015; Litvak ”
A Neural Attention Model for Abstractive Sentence Summarization (2015, background): “Recently Filippova and Altun (2013) developed a strictly extractive system that is trained on a relatively large corpora (250K sentences) of article-title pairs.”
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “Gigaword, along with similar text-headline datasets (Filippova and Altun, 2013), are also used for the related sentence compression task (Dorr et al.”
TreeTalk: Composition and Compression of Trees for Image Descriptions (2014, background): “Much work has considered deletion-only edits like ours (Knight and Marcu, 2000; Turner and Charniak, 2005; Cohn and Lapata, 2007; Filippova and Altun, 2013), while recent ones explore more complex edits, such as substitutions, insertions and reordering (Cohn and Lapata, 2008).”
Neural Text Summarization: A Critical Evaluation (2019, background): “Non-neural approaches (Neto et al., 2002; Dorr et al., 2003; Filippova and Altun, 2013; Col-menares et al., 2015) utilized domain expertise to develop heuristics for summary content selection, whereas more recent, neural techniques allow for end-to-end training.”
Sentence Compression by Deletion with LSTMs (2015, methodology): “That is, one usually drops constituents or prunes dependency edges (Jing, 2000; Knight & Marcu, 2000; McDonald, 2006; Clarke & Lapata, 2008; Berg-Kirkpatrick et al., 2011; Filippova & Altun, 2013).”
Eval all, trust a few, do wrong to none: Comparing sentence generation models (2018, methodology): “We train and evaluate all models on a public corpus consisting of 200,000 sentence summaries extracted from news articles3 (Filippova and Altun, 2013).”
Abstractive text summarization using LSTM-CNN based deep learning (2018, background): “The phrases are composed of several words to express the overall meaning, and it is a key concept in the field of phrase filling [2, 9] and phrase extraction [1].”
Strategies for Structuring Story Generation (2019, methodology): “, from Filippova and Altun (2013)7.”
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization (2021, methodology): “…summarization dataset (Hermann et al., 2015), which makes the generated sentences coherent; (2) followed by Google’s sentence compression dataset (Filippova and Altun, 2013), which limits the generated sequence to a single sentence; and (3) ﬁnally the annotated dataset provided by Demszky et al.…”
UvA-DARE (Digital Academic Repository) A Dataset and Evaluation Metrics for Abstractive Compression of Sentences and Short Paragraphs (unknown): “Filippova and Altun (2013) extracted deletion-based compressions by aligning news headlines to ﬁrst sentences, yielding a corpus of 250,000 parallel sentences.”