Conditional Generation and Snapshot Learning in Neural Dialogue Systems

By Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina Maria Rojas-Barahona, Pei-hao Su, Stefan Ultes, David Vandyke, Susannah Young. 2016

81 citations recorded by the source corpus.

Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential signals by applying a companion cross-entropy objective function to the conditioning vector. The experimental and analytical results demonstrate firstly that competition occurs between the conditioning vector and the LM, and the differing architectures provide different trade-offs between the two. Secondly, the discriminative power and transparency of the conditioning vector is key to providing both model interpretability and better performance. Thirdly, snapshot learning leads to consistent performance improvements independent of which architecture is used.

Subfields: Artificial Intelligence, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Dialogue & Conversational AI

Citations in the corpus

DART: Open-Domain Structured Data Record to Text Generation (2020, background): “…structure such as slot-value pairs (e.g., WikiBio (Lebret et al., 2016), The E2E Dataset (Novikova et al., 2017b), dialog response generation (Wen et al., 2016)) and tables (e.g., RotoWire (Wiseman et al., 2017), WikiTableText (Bao et al., 2018), LogicNLP (Chen et al., 2020a), ToTTo (Parikh…”
Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021, neutral): “…(Budzianowski et al., 2018) that contains 1,839/117/141 dialogues for training/validation/testing, belonging to restaurant, attraction, and hotel domains, and (3) Cam-Rest (Wen et al., 2016) that contains 406/135/135 dialogues from the restaurant reservation domain for training/validation/testing.”
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models (2017, background): “Neural conversation models are the latest development in the domain of conversation modeling, with the promise of training computers to converse in an end-to-end fashion (Vinyals and Le, 2015; Shang et al., 2015; Sordoni et al., 2015; Wen et al., 2016).”
Iterative policy learning in end-to-end trainable task-oriented neural dialog models (2017, neutral): “Note that the results on task success rate in this work should not be directly compared to the numbers in [17, 44], as both the dialog agent and the user simulator in our study are end-to-end models that take noisy natural language utterance as input and directly generate the final dialog act output”
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling (2021, methodology): “[34] Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan 424 Ultes, David Vandyke, and Steve Young.”
Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training (2020, methodology): “Dataset We evaluate the proposed TTOS model on two widely used multi-turn task-oriented dialogue datasets: CamRest (Wen et al., 2016) and In-Car Assistant (Eric and Manning, 2017).”
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention (2019, background): “As pointed in (Wen et al., 2016b), these models though achieving good performance on limited domains, suffer from scalability problem as the possible dialog acts grow combinatorially with the number of domains.”
Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems (2020, methodology): “First, we remove the dialog state memory updating (w/o DSMU), resulting in significant performance degradation on both In-Car Assistant and CamRest datasets.”
Effective Data Augmentation Approaches to End-to-End Task-Oriented Dialogue (2019, methodology): “We used two datasets: CamRest676 [13], [14], [15] and KVRET [5], both of which are manually created by crowd-sourcing workers on the Amazon Mechanical Turk”
Cluster-based beam search for pointer-generator chatbot grounded by knowledge (2020, methodology): “Recently, neural response generation has attracted a lot of attention (Vinyals and Le 2015; Shang, Lu, and Li 2015; Sordoni et al. 2015; Wen et al. 2016).”
Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures (2018, methodology): “…the Sequicity framework, called Two Stage CopyNet ( TSCP ), which has fewer number of parameters and trains faster than state-of-the-art baselines (Wen et al., 2017b, 2016a, 2017a); (3) We demonstrate that TSCP signiﬁcantly outperforms state-of-the-art baselines on two large-scale datasets,…”
Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning (2021, methodology): “We evaluate MCL on three benchmark TDS datasets: Cam-Rest (Wen et al. 2016), In-Car Assistant (Eric and Manning 2017), and Multi-WOZ 2.1 (Budzianowski et al. 2018).”
MOSS: End-to-End Dialog System Framework with Modular Supervision (2019, methodology): “As language model type condition (Wen et al. 2016), kt is concatenated with the word embedding of each aj , j ∈ [1, .”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Task-oriented Dialogue Wen et al. (2016a) Taskmaster Contain dialogues spanning over 6 domains.”