Few-shot Natural Language Generation for Task-Oriented Dialog

By Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, Jianfeng Gao. 2020

210 citations recorded by the source corpus.

As a crucial component in task-oriented dialog systems, the Natural Language Generation (NLG) module converts a dialog act represented in a semantic form into a response in natural language. The success of traditional template-based or statistical models typically relies on heavily annotated data, which is infeasible for new domains. Therefore, it is pivotal for an NLG system to generalize well with limited labelled data in real applications. To this end, we present FewshotWOZ, the first NLG benchmark to simulate the few-shot learning setting in task-oriented dialog systems. Further, we develop the SC-GPT model. It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains. Experiments on FewshotWOZ and the large Multi-Domain-WOZ datasets show that the proposed SC-GPT significantly outperforms existing methods, measured by various automatic metrics and human evaluations.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, background): “Pre-trained models like GPT2 (Radford et al., 2019) have shown great potential to address this challenge (Peng et al., 2020; Chen et al., 2020), and combining pre-trained models with self-training has been shown to improve data efﬁciency even further (Arun et al., 2020).”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “In the more general area of natural language generation, few-shot learning is a growing topic (Chen et al., 2020), e.g., in dialog generation (Peng et al., 2020; Shalyminov et al., 2019).”
A Simple Language Model for Task-Oriented Dialogue (2020, methodology): “[37] used GPT-2 to learn a response generator conditioned on oracle dialogue acts which did not evaluate on dialogue state tracking.”
Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, methodology): “We used SetSUMBT (van Niekerk et al., 2021) as belief tracker and SC-GPT (Peng et al., 2020) as NLG module to accompany the dialogue policies Bin, Gold and DDPT in the dialogue system pipelines.”
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020, background): “The latter is a one-time adaptation process, and the focus is to optimize performance on a target domain transferred from source domains but without considering potential performance drop on source domains (Mi et al., 2019; Qian and Yu, 2019; Peng et al., 2020).”
Text-to-Text Pre-Training for Data-to-Text Tasks (2020, methodology): “• SC-GPT2 (Peng et al., 2020) is a GPT-2 (345M parameters) model that is further pre-trained on a large data-to-text dialog corpus consisting of 400,000 examples and ﬁnally ﬁne-tuned on MultiWoz.”
Evaluation of Text Generation: A Survey (2020, background): “However, the most commonly used human evaluations of dialog systems is still via crowd-sourcing platforms such as Amazon Mechanical Turk (AMT) (Serban et al., 2016a; Peng et al., 2020; Li et al., 2020; Zhou et al., 2020).”
Mitigating Political Bias in Language Models Through Reinforced Calibration (2021, background): “Large-scale language models (LMs) can generate human-like text and have shown promise in many Natural Language Generation (NLG) applications such as dialogue generation (Zhang et al. 2020; Peng et al. 2020) and machine translation (Yang et al. 2020; Zhu et al. 2020).”
Neural Data-to-Text Generation with LM-based Text Augmentation (2021, background): “(Budzianowski and Vulić, 2019; Chen et al., 2020; Peng et al., 2020) tackle the few-shot challenge by finetuning a pretrained LM to incorporate prior knowledge from general-domain text or data-text pairs.”
Few-Shot Named Entity Recognition: A Comprehensive Study (2020, background): “In contrast to NLG, to our best knowledge, large-scale supervised pre-training has been little studied for natural language understanding (NLU).”
Speech-Language Pre-Training for End-to-End Spoken Language Understanding (2021, background): “, GPT), for NLP tasks like natural language understanding, natural language generation, or end-to-end task completion have been explored for dialog systems [16, 17, 18, 19].”
Template Guided Text Generation for Task Oriented Dialogue (2020, methodology): “Following (Peng et al., 2020), they are asked to rate each response on a scale of 1 (bad) to 3 (good) along two axes - informativeness and naturalness.”
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue (2020, background): “Peng et al. (2020) focus on the natural language generation (NLG) task, which assumes dialogue acts and slot-tagging results are given to generate a natural language response.”
Automatically Exposing Problems with Neural Dialog Models (2021, methodology): “Controlled Generation Most previous work in controlled text generation involve training or ﬁne-tuning the whole model (Ficler and Goldberg, 2017; Keskar et al., 2019; Peng et al., 2020; Ziegler et al., 2019).”
Building Adaptive Acceptability Classifiers for Neural NLG (2021, background): “Recently, there have been several efforts to train NLG models in a few-shot setting (Chen et al., 2020; Peng et al., 2020; Arun et al., 2020; Heidari et al., 2021).”
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems (2021, background): “Following Peng et al. (2020b); Kale and Rastogi (2020b), the “Naive” canonical representation A = ∑A i=1 ai(si = vi) is the set of actions produced by the system, where A is the total number of actions for this turn.”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, background): “…et al., 2017; Schuster et al., 2019; Chen et al., 2018; Liu et al., 2019b), and large pre-trained language models are shown to possess the capability to quickly adapt to task-oriented dialogue tasks by using only a few data samples (Peng et al., 2020b; Madotto et al., 2020b; Wu et al., 2020).”