Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

By Massimo Caccia, Pau Rodr'iguez L'opez, Oleksiy Ostapenko, Fabrice Normandin, Min Lin, Lucas Caccia, Issam Hadj Laradji, Irina Rish, Alexander Lacoste, David Vázquez and 1 more. 2020

68 citations recorded by the source corpus.

Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. Two recent continual-learning scenarios have opened new avenues of research. In meta-continual learning, the model is pre-trained to minimize catastrophic forgetting of previous tasks. In continual-meta learning, the aim is to train agents for faster remembering of previous tasks through adaptation. In their original formulations, both methods have limitations. We stand on their shoulders to propose a more general scenario, OSAKA, where an agent must quickly solve new (out-of-distribution) tasks, while also requiring fast remembering. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario. We empirically show that Continual-MAML is better suited to the new scenario than the aforementioned methodologies, as well as standard continual learning and meta-learning approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, background): “The variable z is often referred to as a task (or context) (Caccia et al., 2020; Normandin et al., 2021).”
Continual Learning via Local Module Composition (2021, background): “Unfortunately, in many realistic CL scenarios task identities are unavailable at test time [23, 35, 14].”
Wanderlust: Online Continual Object Detection in the Real World (2021, methodology): “There is an emerging line of works on online continual learning [13, 2, 4, 3, 36].”
New Insights on Reducing Abrupt Representation Change in Online Continual Learning (2021, methodology): “Finally, we report the Averaged Anytime Accuracy (AAA) (Caccia et al., 2020), which measures how well the model performed over the learning experience Computation and Memory Constraints While memory constraints are well documented in previous work, careful monitoring of computation is often…”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Indeed, this phased learning setting can be adopted for continual learning deployment at meta-test time by training the inner loop over a sequence of tasks (Javed and White, 2019; Spigler, 2019; Beaulieu et al., 2020; Caccia et al., 2020; Kim et al., 2021a; Co-Reyes et al., 2021a).”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, background): “This setting reflects many real-world data streams [6, 25] and offers a challenging testbed for online CL research.”
CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions (2020, background): “This is mainly due to the proliferation of different settings only covering partial aspects of the continual learning paradigm, with diverse training and evaluation protocols, metrics and datasets used (Lesort et al., 2020; Caccia et al., 2020).”
Towards mental time travel: a hierarchical memory for reinforcement learning agents (2021, background): “The agents can even extrapolate far beyond the training distribution to recall words after subsequent learning episodes without ever being trained to do so—better memory systems can help with the challenge of bridging from meta-learning to continual learning, which is receiving increasing interest [”
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness (2021, background): “Continual-MAML [12] aims for online fast adaptation to new tasks while accumulating knowledge on old tasks and assume previous tasks can be unlimited revisited.”
Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, methodology): “The second measures the algorithms cumulative performance throughout its life , it is referred to as Online Cumulative Performance (Caccia et al., 2020): It evaluates the performance of “learning while doing” where the goal is the results achieved while learning, i.e. while θ changes and not with a ”
SPeCiaL: Self-Supervised Pretraining for Continual Learning (2021, methodology): “Instead, we assume that at each timestep t , input samples are drawn according to the conditional distribution P ( X t | C t ) , where C t ∈ C is a hidden time-dependent context variable [ Caccia et al. , 2020b ] , de-noting the state of the distribution at time t .”
Learning to Learn and Remember Super Long Multi-Domain Task Sequence (2022, methodology): “Continual meta-learning [1, 13, 54, 74] is to apply the meta-learning techniques for continual learning.”
Scaling the Number of Tasks in Continual Learning (2022, methodology): “The scenario we propose has some similarity with the OSAKA [Caccia et al., 2020] framework.”
Contextual Transformation Networks for Online Continual Learning (2021, background): “Importantly, we note that there exist other continual learning variants that intersect with meta learning, such as meta-continual learning (Javed & White, 2019) and continual-meta learning (He et al., 2019; Caccia et al., 2020).”
Continual Learning with Foundation Models: An Empirical Study of Latent Replay (2022, background): “…( e.g. ImageNet (Deng et al., 2009)), except speciﬁc applications ( e.g. segmentation (Cermelli et al., 2020), meta CL (Caccia and Pineau, 2021; Caccia et al., 2020; Javed and White, 2019)), is rarely used in CL. Hayes et al. (2018); Lesort et al. (2021b); Chrysakis and Moens (2020);…”
Formalizing the Generalization-Forgetting Trade-off in Continual Learning (2021, background): “Common solutions to the forgetting problem involve dynamic architectures and flexible knowledge representation such as [39, 49, 47, 7], regularization approaches including [27, 50, 2] and memory/experience replay [31, 32, 11].”
A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning (2020, methodology): “…for a dynamic Markov chain of other agent policies, we are leveraging an approach that has recently become popular for supervised learning called meta-continual learning (Riemer et al., 2019; Javed & White, 2019; Spigler, 2019; Beaulieu et al., 2020; Caccia et al., 2020; Gupta et al., 2020).”