Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning
By Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias, Idan Szpektor, Craig Boutilier and 1 more. 2022
13 citations recorded by the source corpus.
Despite recent advances in natural language understanding and generation, and decades of research on the development of conversational bots, building automated agents that can carry on rich open-ended conversations with humans"in the wild"remains a formidable challenge. In this work we develop a real-time, open-ended dialogue system that uses reinforcement learning (RL) to power a bot's conversational skill at scale. Our work pairs the succinct embedding of the conversation state generated using SOTA (supervised) language models with RL techniques that are particularly suited to a dynamic action space that changes as the conversation progresses. Trained using crowd-sourced data, our novel system is able to substantially exceeds the (strong) baseline supervised model with respect to several metrics of interest in a live experiment with real users of the Google Assistant.
Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Timo Schick
- Hinrich Schütze
- Jianfeng Gao
- Thomas Scialom
- Yejin Choi
- Asli Celikyilmaz
In Natural Language Processing
- Timo Schick
- Asli Celikyilmaz
- Thomas Scialom
- Hinrich Schütze
- Edouard Grave
- Maria Lomeli
- Yejin Choi
- Jane Dwivedi-Yu
In Reinforcement Learning
- Ramakanth Pasunuru
- Yann LeCun
- Asli Celikyilmaz
- Kang Liu
- Siqi Sun
- Christoforos Nalmpantis
- Roberta Raileanu
- Thomas Scialom
Citations in the corpus
- Augmented Language Models: a Survey (2023, background): “, 2020) to training more helpful, harmless, and accurate assistants (Glaese et al., 2022; Cohen et al., 2022; Ouyang et al., 2022; Bai et al., 2022).”
- A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development (2023, methodology): “Thus, RLHF has been positively applied to fine-tuning language models in most domains in natural language process (NLP) such as dialogue [88], [89], translation [90], [91], story generation [92], evidence extraction [93], [94], and semantic parsing [95].”