Recent advances in deep learning based dialogue systems: a systematic survey
By Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Vishnumurthy Adiga, Erik Cambria. 2021
352 citations recorded by the source corpus.
Dialogue systems are a popular natural language processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this task are carried out, and most of them are deep learning based due to their outstanding performance. In this survey, we mainly focus on the deep learning based dialogue systems. We comprehensively review state-of-the-art research outcomes in dialogue systems and analyze them from two angles: model type and system type. Specifically, from the angle of model type, we discuss the principles, characteristics, and applications of different models that are widely used in dialogue systems. This will help researchers acquaint these models and see how they are applied in state-of-the-art frameworks, which is rather helpful when designing a new dialogue system. From the angle of system type, we discuss task-oriented and open-domain dialogue systems as two streams of research, providing insight into the hot topics related. Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research. Finally, some possible research trends are identified based on the recent research outcomes. To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques. We speculate that this work is a good starting point for academics who are new to the dialogue systems or those who want to quickly grasp up-to-date techniques in this area.
Subfields: Artificial Intelligence, Natural Language Processing, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jason Aaron Edward Weston
- Kurt Shuster
- Mohit Bansal
- Luke S. Zettlemoyer
- Y-Lan Boureau
- Arthur Szlam
- Eric Michael Smith
- Percy Liang
In Natural Language Processing
- Jason Aaron Edward Weston
- Kurt Shuster
- Y-Lan Boureau
- Eric Michael Smith
- Arthur Szlam
- Jing Xu
- Emily Dinan
- Stephen Roller
In Dialogue & Conversational AI
- Kurt Shuster
- Jason Aaron Edward Weston
- Jing Xu
- Eric Michael Smith
- Stephen Roller
- Mojtaba Komeili
- Philip S. Yu
- Arthur Szlam
Citations in the corpus
- BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “Open-domain dialogue models While opendomain dialogue has a rich history (Chen et al., 2017; Gao et al., 2019; Ni et al., 2021) the area has made significant recent progress by pre-training ever-larger neural models.”
- Generative Spoken Dialogue Language Modeling (2022, methodology): “Models trained with either autoencoder objectives (Ondel et al., 2016; van den Oord et al., 2017) or masked objectives (CPC: van den Oord et al., 2018; APC: Chung and Glass, 2020; wav2vec 2.0: Baevski et al., 2020; HuBERT: Hsu et al., 2021a; MockingJay: Liu et al., 2020) from raw speech can learn…”
- ChatGPT: Jack of all trades, master of none (2023, background): “Only 18% of users using the OpenAI API queried GPT-3 model with tasks familiar to typical NLP tasks, most of which are analytical.”
- A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, background): “In general, PLMs-based dialogue systems often comprise multiple sub-modules, like Nature Language Understanding, Dialogue Management, Nature Language Understanding, or Knowledge Introduction modules [109].”
- Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study (2023, background): “These LLMs are designed to process and generate human-like languages, enabling them to perform tasks such as language translation [1], content generation [2], conversational AI [3], etc.”
- Sequence Parallelism: Long Sequence Training from System Perspective (2021, background): “, ∗Equal Contribution 2020) and dialogue system (Ni et al., 2021)).”
- A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, background): “These models can be used for a wide range of NLP tasks such as dialogue systems [58], , translation [59] and question answering [60].”