Source-side Prediction for Neural Headline Generation

By Shun Kiyono, Sho Takase, Jun Suzuki, Naoaki Okazaki, Kentaro Inui, Masaaki Nagata. 2017

9 citations recorded by the source corpus.

The encoder-decoder model is widely used in natural language generation tasks. However, the model sometimes suffers from repeated redundant generation, misses important phrases, and includes irrelevant entities. Toward solving these problems we propose a novel source-side token prediction module. Our method jointly estimates the probability distributions over source and target vocabularies to capture a correspondence between source and target tokens. The experiments show that the proposed model outperforms the current state-of-the-art method in the headline generation task. Additionally, we show that our method has an ability to learn a reasonable token-wise correspondence without knowing any true alignments.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Direct Output Connection for a High-Rank Language Model (2018, methodology): “For the base model, we adopted an encoder-decoder with an attention mechanism described in Kiyono et al. (2017). The encoder consists of a 2-layer bidirectional LSTM, and the decoder consists of a 2-layer LSTM with attention proposed by Luong et al. (2015). We interpreted the layer after computing t”
This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation (2019, methodology): “Furthermore, current methods on news headline generation (Lopyrev, 2015; Tilk and Alumäe, 2017; Kiyono et al., 2017; Tan et al., 2017; Shen et al., 2017) most follow the encoder-decoder model, while our model uses a multi-sentence selection and rewriting framework.”
Fill in the BLANC: Human-free quality estimation of document summaries (2020, background): “It is notable that ROUGE scores are often cited even as applied to automated headline generation [29, 30, 31, 32] where it is hard to imagine that any single headline could be conﬁdently regarded as the best possible headline for a document.”