Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning
By Amal Alabdulkarim, W. Li, Lara J. Martin, Mark O. Riedl. 2021
26 citations recorded by the source corpus.
The advent of large pre-trained generative language models has provided a common framework for AI story generation via sampling the model to create sequences that continue the story. However, sampling alone is insufficient for story generation. In particular, it is hard to direct a language model to create stories to reach a specific goal event. We present two automated techniques grounded in deep reinforcement learning and reward shaping to control the plot of computer-generated stories. The first utilizes proximal policy optimization to fine-tune an existing transformer-based language model to generate text continuations but also be goal-seeking. The second extracts a knowledge graph from the unfolding story, which is used by a policy network with graph attention to select a candidate continuation generated by a language model. We report on automated metrics pertaining to how often stories achieve a given goal event as well as human participant rankings of coherence and overall story quality compared to baselines and ablations.
Subfields: Artificial Intelligence, General Social Sciences, Reinforcement Learning, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yulia Tsvetkov
- Yejin Choi
- Sachin Kumar
- Ximing Lu
- Sean Welleck
- Vidhisha Balachandran
- Antonios Anastasopoulos
- Jack Hessel
In General Social Sciences
- Vidhisha Balachandran
- Yulia Tsvetkov
- Yejin Choi
- Prithviraj Ammanabrolu
- Jack Hessel
- Lianhui Qin
- Antonios Anastasopoulos
In Reinforcement Learning
- Yejin Choi
- Yulia Tsvetkov
- Prithviraj Ammanabrolu
- Liwei Jiang
- Sean Welleck
- Jack Hessel
- Lianhui Qin
- Antonios Anastasopoulos
In Generative Models
Citations in the corpus
- Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “, 2023); RL based fine-tuning (Alabdulkarim et al., 2021; Liu et al., 2021b; Ouyang et al., 2022; Stiennon et al., 2020); Prompt-based learning (Gehman et al.”
- Quark: Controllable Text Generation with Reinforced Unlearning (2022, methodology): “Compared to PPO, Quark achieves better performance, with less parameters and shorter training time.”