MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems

By Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Pascale Fung. 2020

183 citations recorded by the source corpus.

In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to "carryover" the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20\% training data, and 3) Lev greatly improves the inference efficiency.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Lin et al. (2020c) proposed Minimalist Transfer Learning (MinTL) to plug-and-play large-scale pretrained models for domain transfer in dialogue task completion.”
TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems (2020, background): “Recently, there has been growing interest in the so-called end-to-end approach to task-based dialog systems (Peng et al., 2020; Hosseini-Asl et al., 2020; Lin et al., 2020; Wen et al., 2017; Bordes et al., 2016) due to its relatively simple and scalable architecture, and promising results in chatbot”
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling (2021, methodology): “Recently, there has been growing interest in developing deep learning-based end-to-end ToD systems [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] because they can handle complex dialogue patterns with minimal hand-crafted rules.”
Language Models are Few-shot Multilingual Learners (2021, methodology): “…encoder-decoder models, have been used for various sequence generation tasks, such as summarization (Raffel et al., 2020), conversational agents (Lin et al., 2020b,a; Madotto et al., 2020a; Wu and Xiong, 2020; Hosseini-Asl et al., 2020; Lin et al., 2021b), and knowledge grounding (Chen et al.,…”
Dialogue State Tracking with a Language Model using Schema-Driven Prompting (2021, methodology): “Further, our T5-small system is even better than MinTL built on BART-LARGE (Lewis et al., 2020) which has substantially more parameters.”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, unknown): “…2019; Qin et al., 2019, 2020; Raghu et al., 2019; Haihong et al., 2019; He et al., 2020) and hybrid (Shu et al., 2018; Lei et al., 2018; Zhang et al., 2019a; Mehri et al., 2019; Chen et al., 2019; Peng et al., 2020a; Ham et al., 2020; Hosseini-Asl et al., 2020; Le et al., 2020; Lin et al., 2020).”
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022, methodology): “(Budzianowski and Vuli´c, 2019; Hosseini-Asl et al., 2020; Ham et al., 2020; Lin et al., 2020; Yang et al., 2021) utilized pretrained language models such as GPT-2 (Rad-ford et al., 2019) to perform TOD tasks such as language generation or act prediction.”
Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021, background): “Current state-of-the-art models (Chen et al., 2020; Lin et al., 2020; Heck et al., 2020; Hosseini-Asl et al., 2020; Ye et al., 2021; Li et al., 2020) trained with extensive annotated data have been shown promising performance in complex multi-domain conversations (Budzianowski et al.”