POMDP-Based Statistical Spoken Dialog Systems: A Review

By Susannah Young, Milica Gasic, Blaise Thomson, Jason Williams. 2013

1,005 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

CoQA: A Conversational Question Answering Challenge (2018, background): “Due to the conversational nature of questions and answers, our work can also be defined as an instantiation of a dialog problem (Weizenbaum, 1966; Cohen et al., 1982; Young et al., 2013; Serban et al., 2018).”
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, background): “Traditional task-oriented dialogue systems rely on slotfilling and structured modules (e.g., Young et al. (2013); Gao et al. (2019); Jurafsky and Martin (2019)).”
A Simple Language Model for Task-Oriented Dialogue (2020, neutral): “They often use a pipeline approach [46, 58].”
Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018, background): “The most successful goal-oriented dialogue systems model conversation as partially observable Markov decision processes (POMDPs) (Young et al., 2013).”
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (2015, methodology): “Our approach differs from previous work on learning dialogue systems through interaction with humans (Young et al. 2013; Gasic et al. 2013; Cantrell et al. 2012; Mohan and Laird 2014), because it learns off-line through examples of human-human dialogues and aims to emulate the dialogues in the…”
Deep Reinforcement Learning for Dialogue Generation (2016, background): “…Markov Decision Processes (MDPs) (Levin et al., 1997; Levin et al., 2000; Walker et al., 2003; Pieraccini et al., 2009), POMDP (Young et al., 2010; Young et al., 2013; Gašic et al., 2013a; Gašic et al., 2014) models, and models that statistically learn generation rules (Oh and Rudnicky, 2000;…”
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems (2015, background): “Furthermore, the approach does not easily scale to large open domain systems(Young et al., 2013; Gaˇsi´c et al., 2014; Henderson et al., 2014).”
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues (2016, background): “Such models are not specifically designed for the goal-oriented setting, in which dialogue systems were originally developed (Gorin, Riccardi, and Wright 1997; Young 2000; Singh et al. 2002; Young et al. 2013).”
A Network-based End-to-End Trainable Task-oriented Dialogue System (2016, background): “…machine learning approaches to task-oriented dialogue system design have cast the problem as a partially observable Markov Decision Process (POMDP) (Young et al., 2013) with the aim of using reinforcement learning (RL) to train dialogue policies online through interactions with real users (Gašić…”
Toward Controlled Generation of Text (2017, background): “…or probabilistic structured models) on the interpretable latent code, to incorporate prior knowledge or human intentions (Hu et al., 2016a;b); or plug the disentangled generation model into dialog systems to generate natural language responses from structured dialog states (Young et al., 2013).”
Learning End-to-End Goal-Oriented Dialog (2016, methodology): “Traditional dialog systems (Young et al. , 2013) are composed of several modules including a language interpreter, a dialog state tracker and a response generator, usually each is separately hand-crafted or trained.”
A Deep Reinforcement Learning Chatbot (2017, background): “ as a sequential decision making problem based on a Markov decision process (MDP) appeared already in the 1990s for goal-oriented dialogue systems (Singh et al. 1999, 2002, Williams & Young 2007, Young et al. 2013, Paek 2006, Henderson et al. 2008, Pieraccini et al. 2009, Su et al. 2015). One li”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “…language generation (Langkilde and Knight, 1998; Oh and Rudnicky, 2000; Walker et al., 2002; Ratnaparkhi, 2002; Stent et al., 2004; Rieser and Lemon, 2010; Mairesse et al., 2010; Mairesse and Young, 2014; Wen et al., 2015; Sharma et al., 2016), and dialogue policy learning (Young et al., 2013).”
Dialogue Learning With Human-In-The-Loop (2016, neutral): “Efforts include Markov Decision Processes (MDPs) (Levin et al., 1997; 2000; Walker et al., 2003; Pierac-cini et al., 2009), POMDP models (Young et al., 2010; 2013; Gaˇsic et al., 2013; 2014) and policy learning (Su et al., 2016).”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “To make reinforcement learning tractable, it is desired to carefully handle the state and action space [114, 115], which in the end may restrict expressive power and learning capacity of the model.”
Safe Policy Improvement with Baseline Bootstrapping (2017, background): “This scenario is typically encountered when a policy is trained in a simulator and then run in its real environment, for instance in Transfer RL (Taylor & Stone, 2009); or when a system is designed with expert knowledge and then optimized, for example in Dialogue applications (Pietquin et al., 2011;”