Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models

By Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James R. Glass, Fuchun Peng. 2019

9 citations recorded by the source corpus.

In this work, we study how the large-scale pretrain-finetune framework changes the behavior of a neural language generator. We focus on the transformer encoder-decoder model for the open-domain dialogue response generation task. We find that after standard fine-tuning, the model forgets important language generation skills acquired during large-scale pre-training. We demonstrate the forgetting phenomenon through a detailed behavior analysis from the perspectives of context sensitivity and knowledge transfer. Adopting the concept of data mixing, we propose an intuitive fine-tuning strategy named "mix-review". We find that mix-review effectively regularize the fine-tuning process, and the forgetting problem is largely alleviated. Finally, we discuss interesting behavior of the resulting dialogue model and its implications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Optimization & Training

Citations in the corpus

Recipes for Building an Open-Domain Chatbot (2020, background): “as daily dialogue and Cornell Movies (He et al., 2019), and also when multi-tasking across many of these datasets, as we also do here (Shuster et al.”
Continual Learning in Task-Oriented Dialogue Systems (2020, background): “Differently, (Li et al., 2019a) leverage CL for evaluate the quality of chat-bot models and He et al. (2019) studied the catastrophic forgetting problem in Chit-Chat systems.”
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020, background): “To deal with this issue, He et al. (2019) proposed to replay pre-train data during ﬁne-tuning heavily, and Chen et al. (2020) proposed an improved Adam optimizer to recall knowledge captured during pre-training.”
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2019, neutral): “He et al. (2019) recently obtained strong results on the DailyDialog and Cornell Movie tasks in terms of perplexity by pre-training on 10% of CC-NEWS (Bakhtin et al., 2019), thus using 100 million sentences (2.7 billion words) and then ﬁne-tuning a transformer based model with a multi-task strategy.”