Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

By Jun Suzuki, Masaaki Nagata. 2016

87 citations recorded by the source corpus.

This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Convolutional Sequence to Sequence Learning (2017, background): “The current best models on this task are recurrent neural networks which either optimize the evaluation metric (Shen et al., 2016) or address speciﬁc problems of summarization such as avoiding repeated generations (Suzuki & Nagata, 2017).”
MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, background): “…some inputs, including tasks like neural machine translation (NMT) (Cho et al., 2014; Bahdanau et al., 2015a; Vaswani et al., 2017), text summarization (Ayana et al., 2016; Suzuki & Nagata, 2017; Gehring et al., 2017) and conversational response generation (Shang et al., 2015; Vinyals & Le, 2015).”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “Coverage mechanisms (Nallapati et al., 2016; See et al., 2017; Suzuki and Nagata, 2017) and temporal attention (Paulus et al.”
Controllable Abstractive Summarization (2017, methodology): “To address this impediment, (See et al., 2017) introduce coverage modeling, (Paulus et al., 2017) propose intra-decoder attention, and (Suzuki and Nagata, 2017) equip the decoder with an estimator of unigram frequency.”
Classical Structured Prediction Losses for Sequence to Sequence Learning (2017, methodology): “So far we generated candidates with beam search, however, we may also sample to obtain a more diverse set of candidates (Shen et al., 2016 (Ayana et al., 2016), WFE (Suzuki and Nagata, 2017), SEASS (Zhou et al., 2017), DRGD (Li et al., 2017). ong et al., 2015).”
Data-driven Summarization of Scientific Articles (2018, background): “Overlap can be interpreted as the tendency of the model to directly copy input content instead of generating novel correct or incorrect words; whereas Repeat measures a model’s tendency to repeat itself, which is a frequent issue with encoder-decoder models (Suzuki and Nagata, 2017).”
PaperRobot: Incremental Draft Generation of Scientific Ideas (2019, background): “Repetition Removal Similar to many other long text generation tasks (Suzuki and Nagata, 2017), repetition remains a major challenge (Foster and White, 2007; Xie, 2017).”
Source-side Prediction for Neural Headline Generation (2017, methodology): “We selected the settings commonly-used in the previous studies, e.g. , Rush et al. [2015], Nallapati et al. [2016], Suzuki and Nagata [2017].”