On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

By Milica Gasic, Filip Jurcícek, Blaise Thomson, Kai Yu, Susannah Young. 2011

85 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

In Dialogue & Conversational AI

Citations in the corpus

Human-centric Dialog Training via Offline Reinforcement Learning (2020, background): “Improving dialog systems with RL has largely been restricted to task-oriented dialog systems, which have a limited number of task-specific actions (Fatemi et al., 2016; Gašić et al., 2011; Liu and Lane, 2017; Liu et al., 2018; Su et al., 2017).”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “These systems should also be contrasted with systems learning online, such as when the free variables and parameters are optimized directly based on interactions with humans (e.g. Gašić et al. (2011)).”
Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management (2017, methodology): “Initially, systems suffered from slow training, but recent advances in data efficient methods such as Gaussian Processes (GP) have enabled systems to be trained from scratch in on-line interaction with real users (Gašić et al., 2011).”
On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems (2016, background): “Similar to the conclusions drawn in (Gašić et al., 2011), the Subj system suffers from unreliable user feedback.”
POMDP-based dialogue manager adaptation to extended domains (2013, background): “(4) In this way, the stochastic model of theQ-function is effectively transformed into a stochastic policy model, which can be optimised to maximise the reward (Geist and Pietquin, 2011; Gašić et al., 2011; Gašić et al., 2012).”
Learning from real users: rating dialogue success with neural networks for reinforcement learning in spoken dialogue systems (2015, background): “However these ratings were very noisy [12] which lead to slow learning and poor policies [13].”
A Teacher-Student Framework for Maintainable Dialog Manager (2018, methodology): “Recently, reinforcement learning approaches have been applied to optimize dialog systems through interaction with a user simulator or employed real users online (Gaˇsi´c et al., 2011; Su et al., 2016a; Li et al., 2016, Figure 1: An example of a task-oriented dialog after the system comes online.”
Data-Driven Methods for Adaptive Spoken Dialogue Systems (2012, methodology): “5) has shown promising results in online learning with human users [12], and another promising trend of research involves directly learning a policy from data [27].”
POMDP-Based Statistical Spoken Dialog Systems: A Review (2013, methodology): “It has been shown to optimize dialog policies faster than a standard reinforcement learning algorithm [116], and it has been used to successfully train a dialog policy in direct interaction with human users [117].”