Efficient Continual Learning with Modular Networks and Task-Driven Priors

By Tom Véniat, Ludovic Denoyer, Marc'Aurelio Ranzato. 2020

115 citations recorded by the source corpus.

Existing literature in Continual Learning (CL) has focused on overcoming catastrophic forgetting, the inability of the learner to recall how to perform tasks observed in the past. There are however other desirable properties of a CL system, such as the ability to transfer knowledge from previous tasks and to scale memory and compute sub-linearly with the number of tasks. Since most current benchmarks focus only on forgetting using short streams of tasks, we first propose a new suite of benchmarks to probe CL algorithms across these new axes. Finally, we introduce a new modular architecture, whose modules represent atomic skills that can be composed to perform a certain task. Learning a task reduces to figuring out which past modules to re-use, and which new modules to instantiate to solve the current task. Our learning algorithm leverages a task-driven prior over the exponential search space of all possible ways to combine modules, enabling efficient learning on long streams of tasks. Our experiments show that this modular architecture and learning algorithm perform competitively on widely used CL benchmarks while yielding superior performance on the more challenging benchmarks we introduce in this work.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “MNTDP [416] and LMC [318] seek to explicitly find the optimal layout from previous sub-modules and”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, methodology): “MNTDP (Veniat et al., 2020) employed a modular learning approach to transfer knowledge between related tasks while sublinearly scaling with the number of tasks.”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, methodology): “[50] introduced a modular architecture, where each module represents a different atomic skill, and used the REINFORCE algorithm [51] to optimize it.”
Class Gradient Projection For Continual Learning (2022, background): “Prevalent approaches to address the catastrophic forgetting problem in continual learning can be grouped into three broad categories: regularization-based approaches [2, 22, 30, 45], expansion-based approaches [17, 25, 31, 34, 36, 40], and memory-based approaches [3, 27, 33, 38].”
Incremental Task Learning with Incremental Rank Updates (2022, background): “It has been a popular continual learning setup even in the very recent literature [7,14,11,36,31,40].”
TRGP: Trust Region Gradient Projection for Continual Learning (2022, methodology): “To evaluate the forward transfer, we follow the metric used in (Veniat et al., 2020) and consider the accuracy of the model on i -th task after learning the i -th task sequentially, i.e., A i,i as deﬁned in Eq.”
Learning to Prompt for Continual Learning (2021, methodology): “Many recent methods claimed SOTA performance in the simplest task-incremental setting, where task identity is known at test time [19, 45, 57].”
Continual Learning via Local Module Composition (2021, methodology): “Finally, an analysis of longer task sequences (30 and 100 tasks) reveals that LMC tends to spawn much fewer modules to reach good performance than the fully task-aware MNTDP [92] counterpart.”
Wide Neural Networks Forget Less Catastrophically (2021, background): “Expansion-based methods (Rusu et al., 2016; Fernando et al., 2017; Aljundi et al., 2017; Rosenbaum et al., 2018; Chang et al., 2018; Xu and Zhu, 2018; Ferran Alet, 2018; Li et al., 2019; Veniat et al., 2021) add new network modules or experts for new tasks.”
Same State, Different Task: Continual Reinforcement Learning without Interference (2021, background): “By contrast, expansion approaches add new neural resources to enable learning new tasks while preserving components for specific tasks [54, 37, 63].”
Building a Subspace of Policies for Scalable Continual Learning (2022, unknown): “In this work, we introduce an adaptive-size model which strikes a better balance between performance and size, two crucial properties of continual learning systems (Veniat et al., 2020), thus scaling better to long task sequences.”
GROWN: GRow Only When Necessary for Continual Learning (2021, methodology): “…methods (Rusu et al. 2016; Li and Hoiem 2017; Fernando et al. 2017; Rosenfeld and Tsotsos 2018; Hung et al. 2019; Yoon et al. 2017; Li et al. 2019; Veniat, Denoyer, and Ranzato 2020) that dynamically expand the network capacity to reduce the interference between the new tasks and the previously…”
A Probabilistic Framework for Modular Continual Learning (2023, methodology): “Modular approaches to CL [41, 44, 30] represent a deep neural network as a composition of modules, each of which is a parameterized nonlinear transformation, such as a multilayer neural network.”
BNS: Building Network Structures Dynamically for Continual Learning (2021, background): “DEN [61] is similar to MNTDP, although the speciﬁc details are different.”
Continual Learning with Foundation Models: An Empirical Study of Latent Replay (2022, methodology): “…Aljundi et al., 2017), dynamic architectures based methods (Yoon et al., 2017; Schwarz et al., 2018), which include modular methods (Mendez and EATON, 2021; Veniat et al., 2021; Ostapenko et al., 2021), and constrained optimization based methods (Lopez-Paz and Ranzato, 2017; Chaudhry et al., 2018).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Some such approaches introduce heuristics for searching over the space of possible module configurations upon encountering a new task to improve efficiency, for example using programming languages techniques (Valkov et al., 2018) or data-driven heuristics (Veniat et al., 2021).”
Divide and not forget: Ensemble of selectively trained experts in Continual Learning (2024, background): “The class with the highest average softmax is considered as the prediction. settings, e.g., when tasks are unrelated, like in CTrL (Veniat et al., 2020).”