Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems

By Verena Rieser, Oliver Lemon. 2009

105 citations recorded by the source corpus.

We present and evaluate a new model for Natural Language Generation (NLG) in Spoken Dialogue Systems, based on statistical planning, given noisy feedback from the current generation context (e.g. a user and a surface realiser). We study its use in a standard NLG problem: how to present information (in this case a set of search results) to users, given the complex tradeoffs between utterance length, amount of information conveyed, and cognitive load. We set these trade-offs by analysing existing match data. We then train a NLG policy using Reinforcement Learning (RL), which adapts its behaviour to noisy feedback from the current generation context. This policy is compared to several baselines derived from previous work in this area. The learned policy significantly outperforms all the prior approaches.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

The E2E Dataset: New Challenges For End-to-End Generation (2017, background): “Therefore, end-to-end methods have not been able to replicate the rich dialogue and discourse phenomena targeted by previous rule-based and statistical approaches for language generation in dialogue, e.g. (Walker et al., 2004; Stent et al., 2004; Dem-berg and Moore, 2006; Rieser and Lemon, 2009).”
Neural Approaches to Conversational AI (2018, positive): “ to optimize a generation module from corpora (Oh and Rudnicky, 2002; Angeli et al., 2010; Kondadadi et al., 2013; Mairesse and Young, 2014). Most such methods are based on supervised learning, while Rieser and Lemon (2010) takes a decision-theoretic view and uses reinforcement learning to make a tr”
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems (2015, background): “…sentence planning has been explored previously, for example, generating the most likely context-free derivations given a corpus (Belz, 2008) or max-imising the expected reward using reinforcement learning (Rieser and Lemon, 2010), these meth-ods still rely on a pre-existing, handcrafted generator.”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, background): “, 2015), natural language generation (Langkilde and Knight, 1998; Oh and Rudnicky, 2000; Walker et al., 2002; Ratnaparkhi, 2002; Stent et al., 2004; Rieser and Lemon, 2010; Mairesse et al., 2010; Mairesse and Young, 2014; Wen et al., 2015; Sharma et al., 2016), and dialogue policy learning (Young et”
Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking (2015, background): “Statistical approaches have also been studied for sentence planning, for example, generating the most likely context-free derivations given a corpus (Belz, 2008) or maximising the expected reward using reinforcement learning (Rieser and Lemon, 2010).”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, methodology): “This view is shared by many recent approaches based on Reinforcement Learning (rl; Lemon, 2008; Rieser & Lemon, 2009, 2011a), especially those that tackle nlg within a dialogue context.”
Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation (2017, methodology): “Other statistical approaches were proposed us-ing tree-based models and reinforcement learning (Walker et al., 2007; Rieser and Lemon, 2009).”
Natural Language Generation in Dialogue using Lexicalized and Delexicalized Data (2016, methodology): “Previous papers have explored alternative approaches such as corpus-based n-gram models Oh and Rud-nicky, 2002, tree-based models Stent, Prasad, and Walker, 2004, and Reinforcement Learning models Rieser and Lemon, 2010.”
Frames: a corpus for adding memory to goal-oriented dialogue systems (2017, neutral): “summarization, comparison or recommendation (Rieser and Lemon, 2009).”
Training a Natural Language Generator From Unaligned Data (2015, methodology): “While the content planning NLG stage has been used in SDS (e.g., Rieser and Lemon (2010)), we believe that deciding upon the contents of the system’s ut-terance is generally a task for the dialogue manager.”
Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management (2017, background): “…the dialogue history, a dialogue policy (Young et al., 2013; Gaˇsi´c and Young, 2014; Budzianowski et al., 2017) to determine the dialogue ﬂow, and a natural language generator (Rieser and Lemon, 2009; Wen et al., 2015; Hu et al., 2017) to convert conceptual representations into system responses.”
Human-like Natural Language Generation Using Monte Carlo Tree Search (2016, methodology): “As another way to handle the indeterminate characteris- tics of NLG, Lemon (2008; 2011) and Rieser and Lemon (2009) modeled dialog as Markov decision processes (MDPs) and solved them by means of reinforcement learning (Sutton and Barto, 1998).”
Phrase-Based Statistical Language Generation Using Graphical Models and Active Learning (2010, background): “producing a target linguistic style (Paiva and Evans, 2005; Mairesse and Walker, 2008), generating the most likely context-free derivations given a corpus (Belz, 2008), or maximising the expected reward using reinforcement learning (Rieser and Lemon, 2009).”
Neural personalized response generation as domain adaptation (2017, background): “Previous research on taskoriented dialogue generation usually employed handcrafted generator to define the generation decision space with the handcrafted features or statistical models [35, 36, 37, 38, 39, 40, 41, 42].”
The Technology Behind Personal Digital Assistants: An overview of the system architecture and key components (2017, methodology): “There are a number of factors that feed into the LG design, such as information presentation, presenting enough information (to give a good overview of the state of the task) versus keeping the utterances short and understandable, handling error states, and repeated tries [59].”
A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation (2018, methodology): “Earlier work on statistical NLG approaches were typically hybrids of a handcrafted component and a statistical training method (Langkilde and Knight, 1998; Stent et al., 2004; Rieser and Lemon, 2010).”
Hierarchical Reinforcement Learning for Adaptive Text Generation (2010, unknown): “Also, (Lemon, 2008; Rieser and Lemon, 2009) used RL for optimising information presentation styles for search results.”
Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learning (2019, background): “Machine learning for conversational agents has seen great advances (e.g. Tur and Mori, 2011; Gao et al., 2019; Singh et al., 1999; Young et al., 2013; Oh and Rudnicky, 2000; Zen et al., 2009; Reiter and Dale, 2000; Rieser and Lemon, 2010), especially when adopting deep learning models (Deng”
Natural Language Generation as Incremental Planning Under Uncertainty: Adaptive Information Presentation for Statistical Dialogue Systems (2014, methodology): “In this paper we present and evaluate a novel framework for adaptive natural language generation where the problem is formulated as incremental decision making under uncertainty, which can be approached using statistical planning methods [18], [19], [20], [21].”