Continual Classification Learning Using Generative Models

By Frantzeska Lavda, Jason Ramapuram, Magda Gregorová, Alexandros Kalousis. 2018

57 citations recorded by the source corpus.

Continual learning is the ability to sequentially learn over time by accommodating knowledge while retaining previously learned experiences. Neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on previously learned tasks when tasks are presented one at a time. This problem is called catastrophic forgetting. In this work, we propose a classification model that learns continuously from sequentially observed tasks, while preventing catastrophic forgetting. We build on the lifelong generative capabilities of [10] and extend it to the classification setting by deriving a new variational bound on the joint log likelihood, $\log p(x; y)$.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Continual learning: A comparative study on how to defy forgetting in classification tasks (2019, methodology): “DGR [14] PR [47] CCLUGM [48] LGM [49] Constrained”
Rethinking Experience Replay: a Bag of Tricks for Continual Learning (2020, background): “It is also worth mentioning that other works [28], [29] exploit generative models in place of the memory buffer.”
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters (2024, methodology): “Memory-based methods [30, 40, 47, 57, 59, 61, 64] retain the historical knowledge by storing them in a memory bank, which will be accessed and updated in incremental learning.”
Continual Learning in Neural Networks (2019, background): “A generator can be trained to capture the data generating distribution and acts as connectionist memory [118, 81].”
A Continual Learning Survey: Defying Forgetting in Classification Tasks (2019, methodology): “Rehearsal iCaRL [16] ER[49]SER[50] TEM[51]CoPE[33] DGR[12]PR[52] CCLUGM [53] LGM[54] Constrained GEM [55] A-GEM[6]GSS[48] Regularization-basedmethods Prior-focused EWC[27]IMM[28]SI[56] R-EWC[57]MAS[13] [14] Data-focused LwF[58]LFL[59]EBLL[9] DMC[60] Parameter isolation methods”
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain (2020, background): “…either in the form of buffers [Chaudhry et al., 2019, Aljundi et al., 2019a,b, Riemer et al., 2018, Rebuffi et al., 2017, Rolnick et al., 2019] or dedicated generators [Shin et al., 2017, Kamra et al., 2017, Lavda et al., 2018, Lesort et al., 2019b, Atkinson et al., 2018, Kemker and Kanan, 2017].”