A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

By Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minlie Huang. 2020

268 citations recorded by the source corpus.

Story generation, namely, generating a reasonable story from a leading context, is an important but challenging task. In spite of the success in modeling fluency and local coherence, existing neural language generation models (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of long-range coherence in generated stories. We conjecture that this is because of the difficulty of associating relevant commonsense knowledge, understanding the causal relationships, and planning entities and events with proper temporal order. In this paper, we devise a knowledge-enhanced pretraining model for commonsense story generation. We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. To further capture the causal and temporal dependencies between the sentences in a reasonable story, we use multi-task learning, which combines a discriminative objective to distinguish true and fake stories during fine-tuning. Automatic and manual evaluation shows that our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Generation, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yejin Choi
  2. Mohit Bansal
  3. Jianfeng Gao
  4. Percy Liang
  5. Luke S. Zettlemoyer
  6. Noah A. Smith
  7. Zhiyuan Liu
  8. Jason Aaron Edward Weston

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Jianfeng Gao
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Xiaogang Wang
  7. Alan L. Yuille
  8. Jitendra Malik

In Natural Language Generation

  1. Minlie Huang
  2. Marzena Karpinska
  3. Nader Akoury
  4. Xiang Ren
  5. Yejin Choi
  6. Mohit Iyyer
  7. Furu Wei
  8. Bill Yuchen Lin

In Knowledge Graphs & Representation

  1. Xiang Ren
  2. Minlie Huang
  3. Bill Yuchen Lin
  4. Chenguang Zhu
  5. Meng Jiang
  6. Jian Guan
  7. Zhiyuan Liu
  8. Tim Rocktaschel

Citations in the corpus