Progress & Compress: A scalable framework for continual learning

By Jonathan Schwarz, Wojciech M. Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, Raia Hadsell. 2018

1,048 citations recorded by the source corpus.

We introduce a conceptually simple and scalable framework for continual learning domains where tasks are learned sequentially. Our method is constant in the number of parameters and is designed to preserve performance on previously encountered tasks while accelerating learning progress on subsequent problems. This is achieved by training a network with two components: A knowledge base, capable of solving previously encountered problems, which is connected to an active column that is employed to efficiently learn the current task. After learning a new task, the active column is distilled into the knowledge base, taking care to protect any previously acquired skills. This cycle of active learning (progression) followed by consolidation (compression) requires no architecture growth, no access to or storing of previous data or tasks, and no task-specific parameters. We demonstrate the progress & compress approach on sequential classification of handwritten alphabets as well as two reinforcement learning domains: Atari games and 3D maze navigation.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Learning and Evaluating General Linguistic Intelligence (2019, methodology): “For instance, techniques such as elastic weight consolidation (Kirkpatrick et al., 2017) and progress and compress (Schwarz et al., 2018) may hold promise for improving our models’ robustness to forgetting.”
Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “In recent years, interest in continual learning has risen [1, 21, 26, 27, 29, 39, 40, 42, 51], especially from the deep learning research community, due to its potential to reduce training time and training set sizes (e.g., by continuously adapting from previous models), both of which are critical…”
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, methodology): “For comparison, we pick several popular methods (EWC[3], online EWC[23], SI[9], MAS[24], LwF[25]) and state-of-the-art rehearsal-based methods (DGR[8], RtF[14]) with generative model.”
Experience Replay for Continual Learning (2018, methodology): “Likewise, the network and hyperparameters we use are designed to match exactly those used in [29].”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “The importance can be calculated by the Fisher information matrix (FIM), such as EWC [49] and some more advanced versions [50], [104].”
DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “…weight updates in order to maintain knowledge from previous tasks [Kirkpatrick et al., 2017, Zenke et al., 2017, Maltoni and Lomonaco, 2018], or by keeping an old model in memory and distilling knowledge [Hinton et al., 2015] into it later to remember [Li and Hoiem, 2018, Schwarz et al., 2018].”
Policy Consolidation for Continual Reinforcement Learning (2019, background): “In (Schwarz et al., 2018), knowledge is distilled unidirectionally from a flexible network to a more stable one, and vice versa in (Furlanello et al.”
Continual Learning via Sequential Function-Space Variational Inference (2023, methodology): “Our empirical evaluation centers around six sequences of classification tasks: a synthetic sequence of binary-classification tasks with 2D inputs; split MNIST ; split Fashion MNIST ; permuted MNIST ; split CIFAR ; and sequential Omniglot.”
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, methodology): “Finally, we evaluate CAF under the setting of task-incremental learning [58] and compare it to a range of regularization-based methods [7–9, 44, 46, 59, 60].”
Learnability and Algorithm for Continual Learning (2023, methodology): “Many existing approaches have used similar approaches (Ritter et al., 2018; Schwarz et al., 2018; Xu & Zhu, 2018; Castro et al., 2018; Dhar et al., 2019; Lee et al., 2019; Ahn et al., 2019; Liu et al., 2020a).”
A Theoretical Study on Solving Continual Learning (2022, methodology): “Using regularization [12, 13] to minimize changes to model parameters learned from previous tasks is a popular approach [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, methodology): “To provide a thorough evaluation, we investigate how the following existing regularization techniques (coupled with replay strategies such as ER-ACE and DER++) compare with our solution: • Stable SGD (sSGD) [54], which introduces some specific alterations to the model’s training regime that bias the”
Long-Tailed Class Incremental Learning (2022, methodology): “Parameter-isolation methods increase model plasticity by adding more neurons, modules [31,33], branches [23] or masks [26,25,34].”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “For example, regularizationbased methods penalize the changes of important parameters when learning new tasks and typically assign a separate classifier for each task [12,42,64,85,93].”
CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One (2022, neutral): “8) RWALK [7] λ(128) λ(6) λ(48) λ(5) P&C [40] λ(40000) λ(20000) λ(1) λ(80) ∗AFEC [45] λe(1) λe(1) λe(0.”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Several works have performed knowledge transfer across tasks (Ke, Liu, and Huang 2020; Ke et al. 2021; Schwarz et al. 2018; Fernando et al. 2017; Rusu et al. 2016).”
Continual Learning through Retrieval and Imagination (2022, methodology): “…progresses majorly follow two directions: (1) trying to estimate the importance of each network parameter for previous tasks and penalizing the changes of important parameters during the learning of new tasks (Kirkpatrick et al. 2017; Schwarz et al. 2018; Serra et al. 2018; Zenke et al. 2017).”
Transfer without Forgetting (2022, background): “In light of this limitation, the sub-field of Continual Learning (CL) [16,46,63] aspires to train models capable of adaptation and lifelong learning when facing a sequence of changing tasks, either through appositely designed architectures [56,57,38], targeted regularization [34,31,70] or by storing”
ELLE: Efficient Lifelong Pre-training for Emerging Data (2022, background): “(1) First of all, consolidation-based methods (EWC and MAS) perform almost comparable with the naive baseline in either pre-training or downstream tasks.”