A stochastic model of computer-human interaction for learning dialogue strategies

By Esther Levin, Roberto Pieraccini. 1997

Recent progress in the (cid:12)eld of spoken natural language understanding expanded the scope of spoken language systems to include mixed initiative dialogue. Currently there are no agreed upon theoretical foundations for the design of such systems. In this work we propose a stochas- tic model of computer-human interactions. This model can be used for learning and adaptation of the dialogue strategy and for objective evaluation.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

GuessWhat?! Visual Object Discovery through Multi-modal Dialogue (2016, background): “Traditionally, dialogue systems have been built through heavy engineering and hand-crafted expert knowledge, despite machine learning attempts for almost two decades [25, 40].”
Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Data Sets (2008, background): “Previous work has raised the question of whether dialogue management policies can be learned (Levin and Pieraccini 1997) for systems that have only a limited view of the dialogue context, for example, not including prior speech act history (see the following).”
Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management (2017, neutral): “More recently, it has been formulated as a planning problem and solved using reinforcement learning (RL) to optimise a dialogue policy through interaction with users (Levin and Pieraccini, 1997; Roy et al., 2000; Williams and Young, 2007; Jurčı́ček et al., 2011).”
Extending Neural Generative Conversational Model using External Knowledge Sources (2018, background): “Much of the research in dialogue systems from the last few years has focused on replacing all (or some) of its components with Deep Neural Network (DNN) architectures (Levin and Pieraccini, 1997; Dahl et al., 2012; Li et al., 2017; Serban et al., 2016a,b; Vinyals and Le, 2015).”
Transfer Learning for User Adaptation in Spoken Dialogue Systems (2016, background): “Reinforcement Learning [35] is a popular framework for dialogue management in spoken dialogue systems [28, 34].”
Hierarchical Reinforcement Learning for Adaptive Text Generation (2010, background): “However, supervised learning requires a large amount of training data, which may not always be available, and may also produce unpredictable behaviour in cases where a user deviates from the behaviour covered by the corpus (Levin et al., 2000).”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, background): “…dialogue systems have a tracking component that maintains information about the dialogue in a belief state, and a planning component that models the underlying policy, i.e., the selection of actions (Levin and Pieraccini, 1997; Roy et al., 2000; Williams and Young, 2007; Zhang et al., 2020b).”
Estimating the Reliability of MDP Policies: a Confidence Interval Approach (2007, methodology): “Past research into using RL to improve spoken dialogue systems has commonly used Markov Decision Processes (MDP’s) (Sutton and Barto, 1998) to model a dialogue (such as (Levin and Pieraccini, 1997) and (Singh et al., 1999)).”
Interactive reinforcement learning for task-oriented dialogue management (2016, background): “Using reinforcement learning based approaches to train a dialogue manager has been extensively researched (Levin and Pieraccini (1997); Williams (2006); Gašić et al. (2012); Pietquin (2013); Cuayáhuitl (2016); Dhingra et al. (2016)).”
Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning (2019, unknown): “In past work, dialog systems have generally relied on pipeline systems [16, 7].”
Partially observable Markov decision processes for spoken dialog systems (2007, unknown): “…Air Travel domain, Levin et al. constructed a system to optimize the costs of querying the user to restrict (or broaden) their ﬂight search, the costs of presenting too many (or too few) ﬂight options, and the costs of accessing a database (Levin and Pieraccini, 1997; Levin et al., 1998, 2000).”
POMDP-Based Statistical Spoken Dialog Systems: A Review (2013, background): “[155] H. Meng, C. Wai, and R. Pieraccini, ‘‘The use of belief networks for Mixed-Initiative dialog modeling,’’ IEEE Trans.”
Incorporating Unstructured Textual Knowledge Sources into Neural Dialogue Systems (2015, background): “Although initial results are promising, they still require significant additions before replacing more conventional pipeline dialogue systems [8].”
Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems (2021, background): “In planning, the dialogue policy makes decisions at each turn to maximise future rewards at the end of the dialogue (Levin and Pieraccini, 1997).”