Efficient Meta Lifelong-Learning with Limited Memory

By Zirui Wang, Sanket Vaibhav Mehta, Barnab'as P'oczos, Jaime G. Carbonell. 2020

83 citations recorded by the source corpus.

Current natural language processing models work well on a single task, yet they often fail to continuously learn new tasks without forgetting previous ones as they are re-trained throughout their lifetime, a challenge known as lifelong learning. State-of-the-art lifelong language learning methods store past examples in episodic memory and replay them at both training and inference time. However, as we show later in our experiments, there are three significant impediments: (1) needing unrealistically large memory module to achieve good performance, (2) suffering from negative transfer, (3) requiring multiple local adaptation steps for each test example that significantly slows down the inference speed. In this paper, we identify three common principles of lifelong learning methods and propose an efficient meta-lifelong framework that combines them in a synergistic fashion. To achieve sample efficiency, our method trains the model in a manner that it learns a better initialization for local adaptation. Extensive experiments on text classification and question answering benchmarks demonstrate the effectiveness of our framework by achieving state-of-the-art performance using merely 1% memory size and narrowing the gap with multi-task learning. We further show that our method alleviates both catastrophic forgetting and negative transfer at the same time.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Zixuan Ke
3. Hu Xu
4. Lei Shu
5. Bin Liu
6. Gyuhak Kim
7. Tatsuya Konishi
8. Martin Wistuba

In Computer Vision and Pattern Recognition

1. Zixuan Ke
2. Bin Liu
3. Bing Liu
4. Hu Xu
5. Gyuhak Kim
6. Hang Su
7. Jun Zhu
8. B. Ermiş

In Meta & Continual Learning

1. Bing Liu
2. Zixuan Ke
3. Bin Liu
4. Gyuhak Kim
5. Hu Xu
6. Lei Shu
7. Tatsuya Konishi
8. Liyuan Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “MBPA++ [442], EMAR [465], DnR [396], ARPER [298], Total Recall [253]), generative replay (PAGeR [415], LAMOL [395], ACM [499], NER [434]), parameter allocation (TPEM [131]), modular network (ProgModel [377]), meta-learning (MetaMBPA++ [442], MeLL [422], CML [455]), etc.”
• Progressive Prompts: Continual Learning for Language Models (2023, methodology): “We consider nine baseline methods for comparison with ProgressivePrompts: • Finetune (Wang et al., 2020; de Masson D’Autume et al., 2019; Huang et al., 2021): train all model parameters on a sequence of tasks (without adding any regularization constraints or replaying samples from the previous…”
• Continual Learning of Natural Language Processing Tasks: A Survey (2022, background): “Except for TIL and DIL, the performance of systems of other settings are still far from ONE, not to mention MTL [Xia et al., 2021; Wang et al., 2020].”
• Continual Training of Language Models for Few-Shot Learning (2022, methodology): “(2) Replay methods retain (Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017; Wang et al., 2020; Guo et al., 2022) or generate some data of old tasks (Shin et al., 2017; He and Jaeger, 2018) and use them in learning a new task.”
• Continual Sequence Generation with Adaptive Compositional Modules (2022, background): “Comparing to continual learning on text classifi-cation and question answering (Wang et al., 2020; Holla et al., 2020; Huang et al., 2021), continual sequence generation is more challenging, since the output is no longer discrete labels but sequential text data in different styles/domains.”
• Memory Efficient Continual Learning with Transformers (2022, methodology): “[62] Zirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos, and Jaime Carbonell.”
• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, background): “(2) Replay-based approaches , such as those in [45, 36, 4, 63], retain some training data of old tasks and use them in learning a new task.”
• Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, background): “…Mallya and Lazebnik, 2018); (iv) meta-learning-based method, which directly optimizes the knowledge transfer among tasks (Riemer et al., 2019; Obamuyide and Vla-chos, 2019), or learns robust data representations (Javed and White, 2019; Holla et al., 2020; Wang et al., 2020) to alleviate forgetting.”
• Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, methodology): “…and propose the “ P rompt A ugmented Ge nerative R eplay” ( PAGeR ) model where, similar to existing approaches (de Masson d'Autume et al., 2019; Wang et al., 2020; Qian et al., 2021; Xia et al., 2021; Wang et al., 2019; Obamuyide and Vlachos, 2019; Han et al., 2020) we do not store real…”
• Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, background): “Using these sampled data or pseudo data can prevent weights from deviating from previous status (Rolnick et al., 2019; Wang et al., 2020; Lopez-Paz and Ranzato, 2017).”
• Continual Learning with Transformers for Image Classification (2022, methodology): “Existing methods can be roughly categorized as replay-based methods [3, 4, 20, 27, 31] that retain some training data of old tasks and use them in learning a new task to circumvent the issue of CF; regularization-based methods [1, 14, 16] add a regularization term to the loss to consolidate…”
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021, background): “Wang et al. (2020a) improves the utilization efficiency of parameters by introducing a rejuvenation approach.”
• InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022, background): “To address these issues, future work can take inspiration from work related to negative task interference (Wang et al., 2020a; Larson and Leach, 2022), transferability (Vu et al., 2020; Wu et al., 2020b; Xing et al., 2022) and lifelong learning (Wang et al., 2020b).”
• Meta-Learning with Sparse Experience Replay for Lifelong Language Learning (2020, methodology): “All these methods are not yet well-suited for application in real-life scenarios – MbPA++ has slow inference, Meta-MbPA has a higher rate of sampling examples from memory, and other methods require task identifiers and multiple epochs of training.”
• Memory Efficient Continual Learning for Neural Text Classification (2022, methodology): “Existing methods for continual learning (CL) can be roughly categorized as follows: (1) Replay-based methods (LopezPaz & Ranzato, 2017; Rolnick et al., 2018; d’Autume et al., 2019; Chaudhry et al., 2019; Wang et al., 2020) retain some training data of old tasks and use them in learning a new task to”