Entity Commonsense Representation for Neural Abstractive Summarization

By Reinald Kim Amplayo, Seonjae Lim, Seung-won Hwang. 2018

36 citations recorded by the source corpus.

A major proportion of a text summary includes important entities found in the original text. These entities build up the topic of the summary. Moreover, they hold commonsense information once they are linked to a knowledge base. Based on these observations, this paper investigates the usage of linked entities to guide the decoder of a neural text summarizer to generate concise and better summaries. To this end, we leverage on an off-the-shelf entity linking system (ELS) to extract linked entities and propose Entity2Topic (E2T), a module easily attachable to a sequence-to-sequence model that transforms a list of entities into a vector representation of the topic of the summary. Current available ELS’s are still not sufficiently effective, possibly introducing unresolved ambiguities and irrelevant entities. We resolve the imperfections of the ELS by (a) encoding entities with selective disambiguation, and (b) pooling entity vectors using firm attention. By applying E2T to a simple sequenceto-sequence model with attention mechanism as base model, we see significant improvements of the performance in the Gigaword (sentence to title) and CNN (long document to multi-sentence highlights) summarization datasets by at least 2 ROUGE points.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhe Gan
2. Jingjing Liu
3. Mohit Bansal
4. Yu Cheng
5. Jianfeng Gao
6. Yen-Chun Chen
7. Chenguang Zhu
8. Linjie Li

In Molecular Biology

1. Heng Ji
2. Zhe Gan
3. Meng Jiang
4. Yu Cheng
5. Yen-Chun Chen
6. Jingjing Liu
7. Chenguang Zhu

Citations in the corpus

• A Survey of Knowledge-enhanced Text Generation (2020, background): “eq + facts retrieval + RL + GAN M1 be shared between several word types. Incorporating NER tags and POS tags can detect named entities and understand input sequence better, hence, further improve NLG [2, 38, 104, 181]. 4 NLG ENHANCED BY KNOWLEDGE BASE/GRAPH, DEPENDENCY GRAPH 4.1 NLG Enhanced by Know”
• Multi-stage Pretraining for Abstractive Summarization (2019, background): “els for text generation, as such models have been successful at pushing the state-of-the-art ever higher (Rush et al., 2015; Luong et al., 2015; Nallapati et al., 2016; See et al., 2017; Paulus et al., 2017; Amplayo et al., 2018; Gehrmann et al., 2018).”
• Distilling Knowledge Learned in BERT for Text Generation (2019, methodology): “E2T cnn (Amplayo et al., 2018) utilized entities provided by an external entity linking system.”
• Distilling the Knowledge of BERT for Text Generation (2019, methodology): “cally designed for summarization. CGU (Lin et al.,2018) augmented convolutional gating units. FTSum g(Cao et al.,2018b) leveraged extra information extraction and dependency parsing features. E2T cnn(Amplayo et al., 2018) utilized entities provided by an external entity linking system. Re3Sum (Cao e”