Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation

By Dingcheng Li, Zheng Chen, Eunah Cho, Jie Hao, Xiaohu Liu, Xing Fan, Chenlei Guo. 2022

63 citations recorded by the source corpus.

Seq2seq language generation models that are trained offline with multiple domains in a sequential fashion often suffer from catastrophic forgetting. Lifelong learning has been proposed to handle this problem. However, existing work such as experience replay or elastic weighted consolidation requires incremental memory space. In this work, we propose an innovative framework, RMR_DSEthat leverages a recall optimization mechanism to selectively memorize important parameters of previous tasks via regularization, and uses a domain drift estimation algorithm to compensate the drift between different do-mains in the embedding space. These designs enable the model to be trained on the current task while keep-ing the memory of previous tasks, and avoid much additional data storage. Furthermore, RMR_DSE can be combined with existing lifelong learning approaches. Our experiments on two seq2seq language generation tasks, paraphrase and dialog response generation, show thatRMR_DSE outperforms SOTA models by a considerable margin and reduces forgetting greatly.

Subfields: Artificial Intelligence, Natural Language Generation, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hang Su
2. Jun Zhu
3. Wanxiang Che
4. Ting-Wei Liu
5. Liyuan Wang
6. Xingxing Zhang
7. Yiming Cui
8. Richard Zhang

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Jun Zhu
5. Jun-Yan Zhu
6. Simone Scardapane
7. Valerio Marsocci
8. Liyuan Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “The applications of continual learning in NLP are also highly diverse, such as dialogue system [130], [131], [260], [260], [282], [298], [437], text classification [87], [172], [325], [460], sentence generation [246], [298], [499], relation learning [343], [357], [455], [465], neural machine transla”
• Multi-Concept Customization of Text-to-Image Diffusion (2022, background): “This poses a few challenges – first, the model tends to forget [12, 35,52] or change [34,41] the meanings of existing concepts: e.g., the meaning of “moon” being lost when adding the “moongate” concept.”
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting (2020, background): “Many approaches have been proposed to overcome the forgetting problem in various domains, such as machine translation (Miceli-Barone et al., 2017; Thompson et al., 2019) and reading comprehension (Xu et al., 2019).”
• An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022, unknown): “ples typically leads to catastrophic forgetting (Ding et al., 2022; Li et al., 2022).”