Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

By Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting-Wei Liu, Xiangzhan Yu. 2020

270 citations recorded by the source corpus.

Deep pretrained language models have achieved great success in the way of pretraining first and then fine-tuning. But such a sequential transfer learning paradigm often confronts the catastrophic forgetting problem and leads to sub-optimal performance. To fine-tune with less forgetting, we propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Specifically, we propose a Pretraining Simulation mechanism to recall the knowledge from pretraining tasks without data, and an Objective Shifting mechanism to focus the learning on downstream tasks gradually. Experiments show that our method achieves state-of-the-art performance on the GLUE benchmark. Our method also enables BERT-base to achieve better performance than directly fine-tuning of BERT-large. Further, we provide the open-source RecAdam optimizer, which integrates the proposed mechanisms into Adam optimizer, to facility the NLP community.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Continual Knowledge Learning of Language Models (2021, methodology): “RecAdam (Chen et al., 2020) We use the same hyperparameter setting for the optimizer as in Chen et al.”
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020, background): “To deal with this issue, He et al. (2019) proposed to replay pre-train data during ﬁne-tuning heavily, and Chen et al. (2020) proposed an improved Adam optimizer to recall knowledge captured during pre-training.”
Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, background): “Some past work in LLL has demonstrated that discriminative models can be incrementally learnt for a sequence of tasks (Kirkpatrick et al., 2017; Chen et al., 2020; Wang et al., 2019).”
Learning Natural Language Generation with Truncated Reinforcement Learning (2022, background): “Yet, such an approach suffers from several issues (Chen et al., 2020): (i) catastrophic forgetting when a model forgets previously learned knowledge and overfits to target domains, (ii) computational inefficiency from fine-tuning billion-parameters networks, and (iii) the need of supervised…”
Continual Learning of Large Language Models: A Comprehensive Survey (2024, unknown): “The bottom layer, CFT, targets models for final downstream tasks on the consumer side (Section 4.3), where the model needs to be updated after deployment for the specified task.”
Perturbation Augmentation for Fairer NLP (2022, background): “, 2021), a situation we call “catastrophic forgetting”, when something we like gets overwritten (McCloskey and Cohen, 1989; Goodfellow et al., 2013; Chen et al., 2020).”
A Survey of Human-in-the-loop for Machine Learning (2021, methodology): “So these models are employed for more tasks by fine-tuning and performing incredible results [40, 41, 42].”
Improving language models fine-tuning with representation consistency targets (2022, methodology): “In contrast to prior works (Devlin et al., 2018; Aghajanyan et al., 2020a; Chen et al., 2020), we do not search for the optimal learning rate for our method across different tasks but instead fix the optimization settings across all the experiments.”
Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, methodology): “Moreover, we apply an objective shifting (OS) [34] mechanism for more effective TS learning, which enables us to train the student network with both teacher’s predictions and ground truth references.”
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization (2021, methodology): “In this paper, we adopt RecAdam from Chen et al. (2020) for the second phase of pre-training to weaken the catastrophic forgetting issue.”
Ultra Fast Speech Separation Model with Teacher Student Learning (2021, methodology): “To avoid this limitation, we introduce the Objective Shifting mechanism to train the student with both the teacher’s prediction and training datasets [17, 18].”
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models (2022, background): “A number of mechanisms have been proposed for increasing robustness against catastrophic forgetting [28, 29, 30, 31, 32, 33].”
Joint Unsupervised and Supervised Training for Multilingual ASR (2021, background): “For example, ﬁnetuning a pretrained model is prone to catastrophic forgetting [10, 11].”
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks (2022, methodology): “To directly adopt existing ﬁne-tuning methods of LMs for non-language tasks, it is common practice to modify the input/output layers and the loss functions, which may cause undesired behaviors like catastrophic forgetting [66, 74].”
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation (2021, background): “Chen et al. (2020) propose a mechanism to recall the knowledge from pretraining tasks.”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “In addition, there are still challenges, such as catastrophic forgetting and incorrect knowledge editing [254], leaving much room for further research.”
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021, methodology): “Some of them utilize the pretrained weights to regularize the deviation of the fine-tuned model (Lee et al., 2020; Daumé III, 2007; Chen et al., 2020), while others compress the output information (Mahabadi et al., 2021) or injects noise into the input (Jiang et al., 2020; Aghajanyan et al., 2021).”
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models (2022, methodology): “Here RecAdam We implement a regularization-based continual learning method for training large LMs called R EC A DAM (Chen et al., 2020) which places a stronger independent assumption among the model parameters, overcoming the limitations of implementing traditional methods such as EWC (Kirkpatrick e”