Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay

By Mohammad Rostami, Soheil Kolouri, Praveen K. Pilly. 2019

83 citations recorded by the source corpus.

Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding space. We learn an abstract generative distribution in the embedding that allows generation of data points to represent past experience. We sample from this distribution and utilize experience replay to avoid forgetting and simultaneously accumulate new knowledge to the abstract distribution in order to couple the current task with past experience. We demonstrate theoretically and empirically that our framework learns a distribution in the embedding, which is shared across all tasks, and as a result tackles catastrophic forgetting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Zixuan Ke
3. Hu Xu
4. Lei Shu
5. Mohammad Rostami
6. Bin Liu
7. Gyuhak Kim
8. Jun Zhu

In Computer Vision and Pattern Recognition

1. Zixuan Ke
2. Bing Liu
3. Bin Liu
4. Mohammad Rostami
5. Hu Xu
6. Gyuhak Kim
7. Hang Su
8. Jun Zhu

In Reinforcement Learning

1. Zixuan Ke
2. Mohammad Rostami
3. Bing Liu
4. Hu Xu
5. Lei Shu
6. Gyuhak Kim
7. Bin Liu
8. Eric Eaton

In Meta & Continual Learning

1. Bing Liu
2. Zixuan Ke
3. Bin Liu
4. Hu Xu
5. Gyuhak Kim
6. Mohammad Rostami
7. Lei Shu
8. Eric Eaton

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “In contrast, autoencoder-based strategies, such as FearNet [210], SRM [361], CLEER [366], EEC [26], GMR [334] and Flashcards [138], can explicitly control the labels of the generated data, although relatively blurred.”
• Learnability and Algorithm for Continual Learning (2023, methodology): “Pseudoreplay methods generate pseudo replay data for old tasks to serve as the replay data (Kamra et al., 2017; Shin et al., 2017; Wu et al., 2018; Seff et al., 2017; Kemker & Kanan, 2018; Hu et al., 2019; Rostami et al., 2019; Ostapenko et al., 2019).”
• A Theoretical Study on Solving Continual Learning (2022, background): “Some systems learn to generate pseudo training data of old tasks to be used in replay, called pseudo-replay [41, 42, 43, 44, 45, 46, 21, 47, 48].”
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “Some works built generators for previous tasks so that learning is done using a mixed set of real data of the new task and generated data of previous tasks (called pseudo-replay ) (Shin et al., 2017; Kamra et al., 2017; Rostami et al., 2019; Hu et al., 2019).”
• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “The methods in [54, 20, 47, 14] learn data generators and generate old task data for learning a new task.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Rostami et al. [134] trained an encoder to map different tasks into a task-invariant Gaussian Mixture Model (GMM).”
• Detection and Continual Learning of Novel Face Presentation Attacks (2021, methodology): “The idea is to enable the model to generate pseudo-data points that are similar to the data points of the past tasks and use the pseudo-data points for pseudo-rehearsal [33, 28, 27].”
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “The methods in (Shin et al., 2017; Kamra et al., 2017; Rostami et al., 2019; He and Jaeger, 2018) build data generators for previous tasks so that in learning the new task, they can use some generated data for previous tasks to help avoid forgetting.”
• Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “Wasserstein Distance (WD) has been used successfully for domain alignment in the UDA literature (Bhushan Damodaran et al. 2018b; Rostami et al. 2019; Xu et al. 2020; Li et al. 2020; Rostami and Galstyan 2020).”
• Generative Continual Concept Learning (2019, methodology): “Building upon our prior work [17], we develop a new method for generative experience replay to tackle catastrophic forgetting.”
• Continual Learning with Knowledge Transfer for Sentiment Classification (2021, methodology): “The methods in [25, 8, 21, 7] instead took the approach of building data generators for the previous tasks so that in learning the new task, they can use some generated data for previous tasks to help avoid forgetting.”
• CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021, background): “Early works are done under Lifelong Learning (LL) (Silver et al., 2013; Ruvolo and Eaton, 2013; Chen and Liu, 2014).”
• MeLL: Large-scale Extensible User Intent Classification for Dialogue Systems with Meta Lifelong Learning (2021, methodology): “Typical methods include experience replay [16, 20, 40], knowledge distillation [6, 26], transfer learning [5, 19, 39], etc.”
• A Survey on Knowledge Distillation of Large Language Models (2024, background): “…or continue learning process, such as rehearsal, which entails periodically revisiting and training on past data (Kirkpatrick et al., 2017; Rostami et al., 2019; Rolnick et al., 2019), as well as regularization methods like elastic weight consolidation (Lee et al., 2017), or dynamic…”
• Continual Learning Based on OOD Detection and Task Masking (2022, unknown): “Several systems learn to generate pseudo training data of old tasks and use them to jointly train the new task, called pseudoreplay [17,20,25,28,30,49,57,60,62,69,69].”
• Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer (2017, methodology): “Saha et al. (2011) use a task interaction matrix to model task relations online and Dekel et al.”