SimpleDS: A Simple Deep Reinforcement Learning Dialogue System

By Heriberto Cuayáhuitl. 2016

89 citations recorded by the source corpus.

This article presents SimpleDS, a simple and publicly available dialogue system trained with deep reinforcement learning. In contrast to previous reinforcement learning dialogue systems, this system avoids manual feature engineering by performing action selection directly from raw text of the last system and (noisy) user responses. Our initial results, in the restaurant domain, report that it is indeed possible to induce reasonable behaviours with such an approach that aims for higher levels of automation in dialogue control for intelligent interactive systems and robots.

Subfields: Artificial Intelligence, Social Psychology, Reinforcement Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Li Deng
3. Lihong Li
4. Zachary Chase Lipton
5. Mohit Bansal
6. Emma Brunskill
7. Dilek Z. Hakkani-Tür
8. Percy Liang

In Social Psychology

1. Maryam Fazel-Zarandi
2. Jared Casale
3. Li Deng
4. Jianfeng Gao
5. Alborz Geramifard
6. Faisal Ahmed
7. David Whitney
8. Jin Cao

In Reinforcement Learning

1. Emma Brunskill
2. Anima Anandkumar
3. Kamyar Azizzadenesheli
4. Lihong Li
5. Zachary Chase Lipton
6. Jianfeng Gao
7. Peter Henderson
8. Brandon Yang

In Dialogue & Conversational AI

1. Anima Anandkumar
2. Emma Brunskill
3. Kamyar Azizzadenesheli
4. Dilek Z. Hakkani-Tür
5. Maryam Fazel-Zarandi
6. Jared Casale
7. Li Deng
8. Jianfeng Gao

Citations in the corpus

• Surprising Negative Results for Generative Adversarial Tree Search (2018, background): “For example (Cuayáhuitl, 2016; Fatemi et al., 2016; Wen et al., 2016) investigate DRL for dialogue policy learning, with (Lipton et al., 2018) addressing the efficiency of exploration.”
• Policy Networks with Two-Stage Training for Dialogue Systems (2016, background): “Proceedings of the SIGDIAL 2016 Conference, pages 101–110, Los Angeles, USA, 13-15 September 2016. c©2016 Association for Computational Linguistics”
• Learning Robust Dialog Policies in Noisy Environments (2017, methodology): “Cuayahuitl [7] presented an implementation of DQN [23, 24].”
• Efficient Exploration for Dialog Policy Learning with Deep BBQ Networks \& Replay Buffer Spiking (2016, methodology): “…domains such as Atari games [Mnih et al., 2013], board games [Silver et al., 2016] and some promising results on dialog [Fatemi et al., 2016, Cuay´ahuitl, 2016, Williams and Zweig, 2016], we use deep reinforcement learning (DRL), an approach that weds the representational power of deep…”
• Interactive reinforcement learning for task-oriented dialogue management (2016, background): “(2012); Pietquin (2013); Cuayáhuitl (2016); Dhingra et al. (2016)). Recent work (Gašić et al. (2013); Cuayáhuitl et al. (2015)) has shown the success of RL approaches to train dialog managers using little to none supervised training data, but they require careful engineering and domain expertise to ”