Distill and Replay for Continual Language Learning

By Jingyuan Sun, Shaonan Wang, Jiajun Zhang, Chengqing Zong. 2020

41 citations recorded by the source corpus.

Accumulating knowledge to tackle new tasks without necessarily forgetting the old ones is a hallmark of human-like intelligence. But the current dominant paradigm of machine learning is still to train a model that works well on static datasets. When learning tasks in a stream where data distribution may fluctuate, fitting on new tasks often leads to forgetting on the previous ones. We propose a simple yet effective framework that continually learns natural language understanding tasks with one model. Our framework distills knowledge and replays experience from previous tasks when fitting on a new task, thus named DnR (distill and replay). The framework is based on language models and can be smoothly built with different language model architectures. Experimental results demonstrate that DnR outperfoms previous state-of-the-art models in continually learning tasks of the same type but from different domains, as well as tasks of different types. With the distillation method, we further show that it’s possible for DnR to incrementally compress the model size while still outperforming most of the baselines. We hope that DnR could promote the empirical application of continual language learning, and contribute to building human-level language intelligence minimally bothered by catastrophic forgetting.

Subfields: Natural Language Processing, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Xuezhi Wang
2. Yanzhe Zhang
3. Liang Lin
4. Liyuan Wang
5. Jun Zhu
6. Yongbin Li
7. Amirkeivan Mohtashami
8. Diyi Yang

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Jun Zhu
5. Chengwei Qin
6. Diyi Yang
7. Yanzhe Zhang
8. Xuezhi Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “MBPA++ [442], EMAR [465], DnR [396], ARPER [298], Total Recall [253]), generative replay (PAGeR [415], LAMOL [395], ACM [499], NER [434]), parameter allocation (TPEM [131]), modular network (ProgModel [377]), meta-learning (MetaMBPA++ [442], MeLL [422], CML [455]), etc.”
• Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, methodology): “To alleviate the catastrophic forgetting issue, two strategies have been widely applied in many lifelong learning works (Rebuffi et al., 2017; Sun et al., 2020; Cao et al., 2020; Yu et al., 2021): (1) Experience Replay which is to repeatedly optimize the model on the stored previous data in…”
• Continual Sequence Generation with Adaptive Compositional Modules (2022, methodology): “On the basis of LAMOL, knowledge distillation (Chuang et al., 2020; Sun et al., 2020) is shown to be effective via improving knowledge transfer while changing tasks.”
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, background): “These include LAMOL (Sun et al., 2019) and its improvements (Chuang et al., 2020; Sun et al., 2020; Kanwatchara et al., 2021).”
• MEDITRON-70B: Scaling Medical Pretraining for Large Language Models (2023, background): “Distilling replay data into the training mixture has been shown to overcome catastrophic forgetting, a phenomenon where a model incorporating out-of-distribution data forgets its previous training (Sun et al., 2020b).”
• Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, methodology): “It has been observed that this scheme is usually insufficient to constrain the PLM (Sun et al., 2020), due to limited information involved.”
• Towards Quantifiable Dialogue Coherence Evaluation (2021, background): “In recent years, KD has been applied to many specific tasks (Sun et al., 2020; Wei et al., 2019; Kim and Rush, 2016; Sourty et al., 2020).”