Sentence Compression by Deletion with LSTMs

By Katja Filippova, Enrique Alfonseca, Carlos A. Colmenares, Lukasz Kaiser, Oriol Vinyals. 2015

266 citations recorded by the source corpus.

We present an LSTM approach to deletion-based sentence compression where the task is to translate a sentence into a sequence of zeros and ones, corresponding to token deletion decisions. We demonstrate that even the most basic version of the system, which is given no syntactic information (no PoS or NE tags, or dependencies) or desired compression length, performs surprisingly well: around 30% of the compressions from a large test set could be regenerated. We compare the LSTM system with a competitive baseline which is trained on the same amount of data but is additionally provided with all kinds of linguistic features. In an experiment with human raters the LSTMbased model outperforms the baseline achieving 4.5 in readability and 3.8 in informativeness.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Exploring the Limits of Language Modeling (2016, neutral): “, or text summarization (Rush et al., 2015; Filippova et al., 2015).”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, methodology): “For example, (Filippova et al. 2015) use a recurrent neural network (RNN) to delete words from a sentence for sentence compression task.”
Towards Universal Paraphrastic Sentence Embeddings (2015, background): “LSTMs have recently been shown to produce state-of-the-art results in a variety of sequence processing tasks (Chen et al., 2015a; Filippova et al., 2015; Xu et al., 2015c; Belinkov & Glass, 2015; Wang & Nyberg, 2015).”
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (2018, methodology): “Gigaword, along with similar text-headline datasets (Filippova and Altun, 2013), are also used for the related sentence compression task (Dorr et al., 2003; Filippova et al., 2015).”
Multi-Reward Reinforced Summarization with Saliency and Entailment (2018, methodology): “…the task of generating a natural short summary of a long document, is more challenging than the extractive paradigm, which only involves selection of important sentences or grammatical sub-sentences (Jing, 2000; Knight and Marcu, 2002; Clarke and La-pata, 2008; Filippova et al., 2015).”
Globally Normalized Transition-Based Neural Networks (2016, methodology): “We also compare to the sentence compression system from Filippova et al. (2015), a 3-layer stacked LSTM which uses dependency label information.”
Controlling Output Length in Neural Encoder-Decoders (2016, neutral): “, 2004) or statistical (Woodsend et al., 2010; Galanis and Androutsopoulos, 2010; Filippova and Strube, 2008; Filippova and Altun, 2013; Filippova et al., 2015) methods.”
Sequence-to-Sequence Learning as Beam-Search Optimization (2016, background): “, 2014), roughly the same model and training has also proven to be useful for sentence compression (Filippova et al., 2015), parsing (Vinyals et al.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “Early summarization works mostly focused on extractive and compression based methods (Jing and McKeown, 2000; Knight and Marcu, 2000; Clarke and Lapata, 2010; Berg-Kirkpatrick et al., 2011; Filippova et al., 2015).”
Optimizing Statistical Machine Translation for Text Simplification (2016, background): “…2007; Narayan and Gardent, 2014; An-grosh et al., 2014) and deletion (Knight and Marcu 2002; Clarke and Lapata 2006; Filippova and Strube 2008; Filippova et al. 2015; Rush et al. 2015; and others) have been intensively studied, there has been considerably less research on developing new…”
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation (2018, background): “As compared to extractive or compressive summarization (Jing and McKeown, 2000; Knight and Marcu, 2002; Clarke and Lapata, 2008; Filippova et al., 2015; Henß et al., 2015), abstractive summaries are based on rewriting as opposed to selecting.”
Neural Headline Generation with Minimum Risk Training (2016, background): “[5] proposes a recurrent neural network with long short term memory (LSTM) [7] for headline generation.”
Controlling the Amount of Verbatim Copying in Abstractive Summarization (2019, background): “The sentences can be optionally compressed to remove inessential phrases, leading to compressive summaries (Martins and Smith 2009; Li et al. 2013; Wang et al. 2013; Filippova et al. 2015; Durrett, Berg-Kirkpatrick, and Klein 2016).”
QuickEdit: Editing Text & Translations by Crossing Words Out (2017, background): “Related to our work, Filippova et al. (2015) considers the task of predicting which tokens can be removed from a sentence without modifying its meaning relying on a recurrent neural network.”