Global Encoding for Abstractive Summarization

By Junyang Lin, Xu Sun, Shuming Ma, Qi Su. 2018

153 citations recorded by the source corpus.

In neural abstractive summarization, the conventional sequence-to-sequence (seq2seq) model often suffers from repetition and semantic irrelevance. To tackle the problem, we propose a global encoding framework, which controls the information flow from the encoder to the decoder based on the global information of the source context. It consists of a convolutional gated unit to perform global encoding to improve the representations of the source-side information. Evaluations on the LCSTS and the English Gigaword both demonstrate that our model outperforms the baseline models, and the analysis shows that our model is capable of generating summary of higher quality and reducing repetition.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Houfeng Wang
2. Xuancheng Ren
3. Shuming Ma
4. Rui Wang
5. Xu Sun
6. Yang Gao
7. Yuxiang Zhou
8. Junyang Lin

Citations in the corpus

• PALM: Pre-training an Autoencoding&autoregressive Language Model for Context-conditioned Generation (2020, neutral): “, 2017); b (Lin et al., 2018); c (Suzuki and Nagata, 2017); d (Cao et al.”
• Distilling the Knowledge of BERT for Text Generation (2019, unknown): “CGU (Lin et al., 2018) augmented convolutional gating units.”
• Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation (2018, background): “However, the sequenceto-sequence model tends to generate short, repetitive (Lin et al., 2018), and dull text (Luo et al.”
• Distilling Knowledge Learned in BERT for Text Generation (2019, unknown): “CGU (Lin et al., 2018) augmented convolutional gating units.”
• Concept Pointer Network for Abstractive Summarization (2019, background): “CGU (Lin et al., 2018) sets a convolutional gated unit and self-attention for global encoding.”
• BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization (2019, background): “…for these models during decoding is simply the source article representations from the encoder and the generated words from the previous time steps (Nallapati et al., 2016; Gu et al., 2016; Lin et al., 2018), while the previous words are also generated based on the article representations.”
• SGM: Sequence Generation Model for Multi-label Classification (2018, background): “, 2017), abstractive summarization (Rush et al., 2015; Lin et al., 2018), style transfer (Shen et al.”