Selective Encoding for Abstractive Sentence Summarization

By Qingyu Zhou, Nan Yang, Furu Wei, M. Zhou. 2017

268 citations recorded by the source corpus.

We propose a selective encoding model to extend the sequence-to-sequence framework for abstractive sentence summarization. It consists of a sentence encoder, a selective gate network, and an attention equipped decoder. The sentence encoder and decoder are built with recurrent neural networks. The selective gate network constructs a second level sentence representation by controlling the information flow from encoder to decoder. The second level representation is tailored for sentence summarization task, which leads to better performance. We evaluate our model on the English Gigaword, DUC 2004 and MSR abstractive sentence summarization datasets. The experimental results show that the proposed selective encoding model outperforms the state-of-the-art baseline models.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Direct Output Connection for a High-Rank Language Model (2018, methodology): “We believe that this is because we adopted a larger embedding size than Zhou et al. (2017).”
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (2018, methodology): “We compare with four abstrac-tive baselines ( s2s-att , PG , SEASS and DRGD ) and the gold summary (Gold).”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “Finally, there are some loosely-related recent works: Zhou et al. (2017) proposed selective gate to improve the attention in abstractive summarization.”
Classical Structured Prediction Losses for Sequence to Sequence Learning (2017, methodology): “So far we generated candidates with beam search, however, we may also sample to obtain a more diverse set of candidates (Shen et al., 2016 (Ayana et al., 2016), WFE (Suzuki and Nagata, 2017), SEASS (Zhou et al., 2017), DRGD (Li et al., 2017). ong et al., 2015).”
Faithful to the Original: Fact Aware Neural Abstractive Summarization (2017, methodology): “Similar to (Zhou et al. 2017), we evaluate the model performance on the development set for every 2000 batches and halve the learning rate if the cost increases for 10 consecutive validations.”
S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension (2017, methodology): “The sequence-to-sequence model is widely-used in many tasks such as machine translation (Luong et al., 2015), parsing (Vinyals et al., 2015b), response generation (Gu et al., 2016), and summarization generation (Zhou et al., 2017).”
Learning to Encode Text as Human-Readable Summaries using Generative Adversarial Networks (2018, methodology): “Compared with the previous work (Zhou et al., 2017), we used simpler model and smaller vocabulary size.”
Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention (2019, background): “…training procedures on the sentence level (Ayana et al., 2016; Li et al., 2018; Edunov et al., 2018; Wang et al., 2018), topic enhancement in the summaries (Wang et al., 2018), additional selective gate networks in the encoder (Zhou et al., 2017), and facts fusion measures (Cao et al., 2018).”
Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation (2018, background): “Text generation involves a wide variety of tasks and studies, such as text summarization [23], [24], machine translation [25], [26], dialogue systems [27], [28], and image captioning [29], [30].”
Concept Pointer Network for Abstractive Summarization (2019, background): “However, more recently, seq2seq models with attention have played a more dominant role in generating abstractive summaries (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; Zhou et al., 2017).”
DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks (2019, background): “Specifically, neural sequenceto-sequence models [22, 2] have led to great progress in important downstream NLP tasks like text summarization [17, 4, 13, 12, 18, 23, 30], machine translation [11, 2, 3, 22], and reading comprehension [28].”
Entity Commonsense Representation for Neural Abstractive Summarization (2018, methodology): “…and statistical features in the encoder, Luong-NMT (Luong et al., 2015) is a two-layer LSTM encoder-decoder model, RAS-Elman (Chopra et al., 2016) uses an attentive CNN encoder and an Elman RNN decoder, and SEASS (Zhou et al., 2017) uses BiGRU encoders and GRU decoders with selective encoding.”
Mixture Content Selection for Diverse Sequence Generation (2019, methodology): “S ELECTOR The GRU has the same size as the generator encoder, and the dimension of expert embedding e z is 300 for NQG++, and 128 for PG.”
SEQˆ3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression (2019, background): “This greatly reduces variance due to sampling in early stages of training, alleviating the need to pretrain individual ABS (Rush et al., 2015) 29.55 11.32 26.42 SEASS (Zhou et al., 2017) 36.15 17.54 33.63 words-lvt5k-1sent (Nallapati et al., 2016) 36 Model R-1 R-2 R-L T OPIARY (Zajic et al., 2007) 2”
Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models (2020, methodology): “Several studies have proposed the combination of a token-level saliency model and a seq-to-seq model, which is not pre-trained, and reported its effectiveness (Gehrmann et al., 2018; Zhou et al., 2017).”
Generating Live Soccer-Match Commentary from Play Data (2019, methodology): “In particular, we introduce a gate mechanism (Zhou et al. 2017), in which the internal representation h computed from the input events by Equation (2) in the section of Encoding events. is used as the aggregated information of the play data sequence, to calculate the importance score γ i for event…”
Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization (2018, methodology): “In Table 1, we report the ROUGE F1 score of our model and the baseline methods on the English Gigaword test set provided by Zhou et al. (2017).”
Global Encoding for Abstractive Summarization (2018, background): “Zhou et al. (2017) pointed out that there is no obvious alignment relationship between the source text and the target summary, and the encoder outputs contain noise for the attention.”
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021, background): “, 2018) or by improving the representation of relevant input tokens (Zhou et al., 2017).”