Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

By Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, Caiming Xiong. 2019

532 citations recorded by the source corpus.

Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks. Despite recent remarkable progress in state-of-the-art deep learning, deep neural networks (DNNs) are still plagued with the catastrophic forgetting problem. This paper presents a conceptually simple yet general and effective framework for handling catastrophic forgetting in continual learning with DNNs. The proposed method consists of two components: a neural structure optimization component and a parameter learning and/or fine-tuning component. By separating the explicit neural structure learning and the parameter estimation, not only is the proposed method capable of evolving neural structures in an intuitively meaningful way, but also shows strong capabilities of alleviating catastrophic forgetting in experiments. Furthermore, the proposed method outperforms all other baselines on the permuted MNIST dataset, the split CIFAR100 dataset and the Visual Domain Decathlon dataset in continual learning setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “…to work better in multitask settings (Ma et al., 2018a; Houlsby et al., 2019) in order to strike the right balance between shared and task-speciﬁc capacity, or learning network structure in order to maximally exploit task relatedness (Li et al., 2019) are exciting avenues for future research.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “reinforcement learning (RCL [464], BNS [344]), architecture search (LtG [251], BNS [344]), variational Bayes (BSA [230]), etc.”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “…Zhang et al., 2022; Wang et al., 2022a; Saha & Roy, 2023), expansion-based methods (Rusu et al., 2016; Yoon et al., 2018; Mallya & Lazebnik, 2018; Li et al., 2019; Hung et al., 2019; Ramesh & Chaudhari, 2022; Douillard et al., 2022), or combinations thereof (Yan et al., 2021; Xie et al., 2022;…”
Class Gradient Projection For Continual Learning (2022, methodology): “Prevalent approaches to address the catastrophic forgetting problem in continual learning can be grouped into three broad categories: regularization-based approaches [2, 22, 30, 45], expansion-based approaches [17, 25, 31, 34, 36, 40], and memory-based approaches [3, 27, 33, 38].”
Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer (2022, methodology): “In dynamic architecture methods [55, 52, 31, 1, 41], the network architecture is modified by expansion or masking when encountering new tasks during learning.”
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, background): “As presented in the main paper, there exist some expansion-based continual learning methods like [69, 34, 17, 59, 25, 19, 39].”
Balancing Stability and Plasticity through Advanced Null Space in Continual Learning (2022, methodology): “Architecture-based methods allocate diﬀerent parameters or add new parameters for the new task, while sharing parameters across tasks to reduce the interference of previous tasks [3, 22, 27–29, 41, 42, 44, 53, 56, 60].”
Online Continual Learning with Contrastive Vision Transformer (2022, neutral): “As new tasks arrive, expansion-based methods dynamically expand networks and keep sub-networks related to previous tasks fixed [69,89,1,59,86,45,82,53].”
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, methodology): “These methods can be further categorized as expanding the model [51, 65, 26, 31, 59], or dividing the model [35, 52, 58, 17, 11].”
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning (2022, methodology): “…learning algorithms, ranging from the dynamic architectures (Rusu et al., 2016; Serra et al., 2018; von Oswald et al., 2020; Yoon et al., 2018; Li et al., 2019; Xu & Zhu, 2018), experience replay (Lin, 1992; Chaudhry et al., 2019b; Aljundi et al., 2019a; Riemer et al., 2019; Rolnick et al.,…”
Probing Representation Forgetting in Supervised and Unsupervised Continual Learning (2022, background): “One class of methods focuses on bypassing this problem by growing architectures over time as new tasks arrive [2,25,41,43].”
The Challenges of Continuous Self-Supervised Learning (2022, methodology): “This is often accomplished either by progressively growing the model each time a new task is added [37, 69, 83], or adapting a common backbone model to each task separately using small task-specific adaptation blocks [41, 51, 67].”
Continual Sequence Generation with Adaptive Compositional Modules (2022, methodology): “Li et al. (2019) employ neural architecture search (Liu et al., 2018) to optimize on new task’s structure before learning new tasks.”
TRGP: Trust Region Gradient Projection for Continual Learning (2022, methodology): “In order to ﬁnd the optimal structure for each of the sequential tasks, Reinforced Continual Learning (RCL) (Xu & Zhu, 2018) leverages reinforcement learning and (Li et al., 2019) adapts architecture search.”
Architecture Matters in Continual Learning (2022, background): “There have been some eﬀorts on applying architecture search to continual learning (Xu and Zhu, 2018; Huang et al., 2019; Gao et al., 2020; Li et al., 2019; Lee et al., 2021).”
Continual Learning with Recursive Gradient Optimization (2022, background): “Since previous samples cannot be accessed to recalculate the gradient, common gradient-based methods (Li et al., 2019; Azizan et al., 2019; Farajtabar et al., 2019) assume that the joint optimal parameter lies in the neighborhood of the previous optimal parameter and use previously calculated gradie”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “…and Lazebnik, 2018), Diffusion-based Neuromodulation (Velez and Clune, 2017), IMM (Lee et al., 2017), Expandable Networks (Yoon et al., 2018; Li et al., 2019), RPSNet (Rajasegaran et al., 2019), reinforcement learning (Kaplanis et al., 2019; Rolnick et al., 2019), and meta-learning (Javed…”
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion (2021, background): “Recent works [76, 46, 33, 23, 27, 64] dynamically expand the network architectures [76, 46] or re-arrange their internal structures [23, 64, 33, 27].”