Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
By Qingyang Wu, Yichi Zhang, Yu Li, Zhou Yu. 2019
63 citations recorded by the source corpus.
Existing dialog system models require extensive human annotations and are difficult to generalize to different tasks. The recent success of large pre-trained language models such as BERT and GPT-2 (Devlin et al., 2019; Radford et al., 2019) have suggested the effectiveness of incorporating language priors in down-stream NLP tasks. However, how much pre-trained language models can help dialog response generation is still under exploration. In this paper, we propose a simple, general, and effective framework: Alternating Recurrent Dialog Model (ARDM). ARDM models each speaker separately and takes advantage of the large pre-trained language model. It requires no supervision from human annotations such as belief states or dialog acts to achieve effective conversations. ARDM outperforms or is on par with state-of-the-art methods on two popular task-oriented dialog datasets: CamRest676 and MultiWOZ. Moreover, we can generalize ARDM to more challenging, non-collaborative tasks such as persuasion. In persuasion tasks, ARDM is capable of generating human-like responses to persuade people to donate to a charity.
Subfields: Natural Language Processing, LLM Agents & Tool Use
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Andrea Madotto
- Pascale Fung
- Zhaojiang Lin
- Jianfeng Gao
- Richard Socher
- Bryan McCann
- Chien-Sheng Wu
- Caiming Xiong
In LLM Agents & Tool Use
Citations in the corpus
- A Simple Language Model for Task-Oriented Dialogue (2020, methodology): “For reference, oracle settings compare across a variety of settings against HDSA [9], ARDM [56], LaRL [63], PARG [16] can be found in the Supplementary Materials, but these comparisons are not essential for end-to-end contributions.”
- SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model (2020, methodology): “However, ARDM is infeasible to track dialog states and requires a separately trained state tracker to accomplish tasks.”
- MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020, background): “Building robust task-oriented dialogue systems is challenging due to complex system design and limited availability of human-annotated data (Wen et al., 2017; Wu et al., 2019b).”