The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management

By Susannah Young, Milica Gasic, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, Kai Yu. 2010

538 citations recorded by the source corpus.

This paper explains how Partially Observable Markov Decision Processes (POMDPs) can provide a principled mathematical framework for modelling the inherent uncertainty in spoken dialogue systems. It briefly summarises the basic mathematics and explains why exact optimisation is intractable. It then describes in some detail a form of approximation called the Hidden Information State model which does scale and which can be used to build practical systems. A prototype HIS system for the tourist information domain is evaluated and compared with a baseline MDP system using both user simulations and a live user trial. The results give strong support to the central contention that the POMDP-based framework is both a tractable and powerful approach to building more robust spoken dialogue systems.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

Citations in the corpus

A Diversity-Promoting Objective Function for Neural Conversation Models (2015, background): “This stands in contrast to conventional dialog systems, which typically are template- or heuristicdriven even where there is a statistical component (Levin et al., 2000; Oh and Rudnicky, 2000; Ratnaparkhi, 2002; Walker et al., 2003; Pieraccini et al., 2009; Young et al., 2010; Wang et al., 2011; Ban”
Neural Approaches to Conversational AI (2018, neutral): “” Since exact optimization in POMDPs is computationally intractable, authors have studied approximation techniques (Roy et al., 2000; Williams and Young, 2007; Young et al., 2010; Li et al., 2009; Gašić and Young, 2014) and alternative representations such as the information states framework (Larsso”
A Persona-Based Neural Conversation Model (2016, background): “…relies extensively on hand-coded rules, typically either building statistical models on top of heuristic rules or templates (Levin et al., 2000; Young et al., 2010; Walker et al., 2003; Pieraccini et al., 2009; Wang et al., 2011) or learning generation rules from a minimal set of authored…”
A Network-based End-to-End Trainable Task-oriented Dialogue System (2016, background): “Furthermore, to make RL tractable, the state and action space must be carefully designed (Young et al., 2013; Young et al., 2010), which may restrict the expressive power and learnability of the model.”
Dialogue Learning With Human-In-The-Loop (2016, neutral): “Efforts include Markov Decision Processes (MDPs) (Levin et al., 1997; 2000; Walker et al., 2003; Pierac-cini et al., 2009), POMDP models (Young et al., 2010; 2013; Gaˇsic et al., 2013; 2014) and policy learning (Su et al., 2016).”
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge (2019, methodology): “…complexities.4 We then implemented several automatic validation procedures for filtering the crowdsourced data in order to ad- 2 Most dialogue systems also include a general intent of the utterance, such as inform, confirm, or request (Young et al. 2010; Wen et al. 2015b; Liu and Lane 2016).”
Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings (2016, background): “The input to our generator are dialogue acts (DA) (Young et al., 2010) representing an action, such as inform or request, along with one or more attributes (slots) and their values.”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “To make reinforcement learning tractable, it is desired to carefully handle the state and action space [114, 115], which in the end may restrict expressive power and learning capacity of the model.”
Word-Based Dialog State Tracking with Recurrent Neural Networks (2014, background): “In POMDP-based systems, the full distribution is considered and then the shape of the distribution as measured by an L2 norm is equally important (Young et al., 2009).”
A Context-aware Natural Language Generator for Dialogue Systems (2016, background): “The function of natural language generation (NLG) components in task-oriented SDS typically is to produce a natural language sentence from a dialogue act (DA) (Young et al., 2010) representing an action, such as inform or request, along with one or more attributes (slots) and their values (see Fig.”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, background): “A statistical dialog system maintains a distribution over multiple hypotheses of the true dialog state, facing with noisy conditions and ambiguity [104].”
Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning (2022, background): “Statistical research focuses on task-oriented dialogues and uses MDPs [18, 35, 36, 41] or POMDPs [42, 44].”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Statistical methods (Lee, 2013; Lee and Eskenazi, 2013; Ren et al., 2013; Williams, 2013, 2014) also suffer from noisy conditions and ambiguity (Young et al., 2010).”
A Context-aware Natural Language Generation Dataset for Dialogue Systems (2016, background): “We understand the task of NLG in the context of taskoriented SDS which use dialogue acts (DA) to represent meaning (Young et al., 2010; Jurčíček et al., 2014).”