Generative Deep Neural Networks for Dialogue: A Short Review

By Iulian Serban, Ryan J. Lowe, Laurent Charlin, Joelle Pineau. 2016

96 citations recorded by the source corpus.

Researchers have recently started investigating deep neural networks for dialogue applications. In particular, generative sequence-to-sequence (Seq2Seq) models have shown promising results for unstructured tasks, such as word-level dialogue response generation. The hope is that such models will be able to leverage massive amounts of data to learn meaningful natural language representations and response generation strategies, while requiring a minimum amount of domain knowledge and hand-crafting. An important challenge is to develop models that can effectively incorporate dialogue context and generate meaningful and diverse responses. In support of this goal, we review recently proposed models based on generative encoder-decoder neural network architectures, and show that these models have better ability to incorporate long-term dialogue history, to model uncertainty and ambiguity in dialogue, and to generate responses with high-level compositional structure.

Subfields: Artificial Intelligence, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Generative Models

Citations in the corpus

CoQA: A Conversational Question Answering Challenge (2018, background): “Sequence-to-sequence (seq2seq) models have shown promising results for generating conversational responses (Vinyals and Le, 2015; Serban et al., 2016; Zhang et al., 2018).”
Wizard of Wikipedia: Knowledge-Powered Conversational agents (2018, neutral): “The current state-of-the-art approaches, sequence to sequence models of various kinds (Sutskever et al., 2014; Vinyals & Le, 2015; Serban et al., 2016; Vaswani et al., 2017) attempt to address some of these skills, but generally suffer from an inability to bring memory and knowledge to bear; as…”
Retrieve and Refine: Improved Sequence Generation Models For Dialogue (2018, background): “However, obtaining good performance on dialogue tasks, where the context still allows many interpretations, remains an open problem despite much recent work (Serban et al., 2016).”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “Bias is Amplified in Generation Existing Transformer generative dialogue models (Serban et al., 2016; Yang et al., 2018; Urbanek et al., 2019) are trained to take as input the dialogue context and generate the next utterance.”
Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018, background): “Still, conversing with such generic chit-chat models for even a short amount of time quickly exposes their weaknesses (Serban et al., 2016; Vinyals and Le, 2015).”
The Second Conversational Intelligence Challenge (ConvAI2) (2019, background): “Conversing with current chit-chat models for even a short amount of time quickly exposes their weaknesses [2, 3].”
What makes a good conversation? How controllable attributes affect human judgments (2019, background): “Well known problems, such as the genericness and repetitiveness of responses (Serban et al., 2016a), remain without a de facto solution.”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “…dialogue system architectures based on neural networks have shown promising results on several dialogue tasks (Ritter et al., 2011; Vinyals and Le, 2015; Lowe et al., 2015a; Sordoni et al., 2015b; Shang et al., 2015; Li et al., 2015; Serban et al., 2016; Serban et al., 2017b,a; Dodge et al., 2015).”
MIRIAM: a multimodal chat-based interface for autonomous systems (2017, background): “There has been a recent upsurge in automated conversational agents or chatbots, performing a variety of functions such as resolving users’ technical problems [5]; performing tasks in the real world (e.”
Generative Spoken Dialogue Language Modeling (2022, background): “Since the early work on end-to-end neural dialogue generation (Vinyals and Le, 2015; Li et al., 2015; Serban et al., 2016), em-powered by scalable methods for language representation (Radford et al., 2018; Lewis et al., 2019), there has been enormous progress in the area of dialogue generation…”
Augmenting Transformers with KNN-Based Composite Memory for Dialog (2020, neutral): “On the modeling side, work has explored both generative (Serban et al., 2016a,b) and retrieval based models (Zhang et al., 2018), which identify the best ut-terance from the training set to return as the dialogue response.”
Measuring and Narrowing the Compositionality Gap in Language Models (2022, methodology): “…dialogue modeling (Zhang et al., 2018; Dinan et al., 2019; Freitas et al., 2020; Roller et al., 2021), which is in turn based on previous success in applying neural networks to the dialogue task (Shang et al., 2015; Sordoni et al., 2015; Vinyals and Le, 2015; Li et al., 2016; Serban et al., 2016).”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, unknown): “, 2018) into chit-chat (Vinyals and Le, 2015; Serban et al., 2016) and task-oriented (Williams and Young, 2007; Young et al.”
A Repository of Conversational Datasets (2019, methodology): “For instance, the datasets are suitable for training generative models of conversational response (Serban et al., 2016; Ritter et al., 2011; Vinyals and Le, 2015; Sordoni et al., 2015; Shang et al., 2015; Kannan et al., 2016), as well as discriminative methods of conversational response selection (L”