Meta-Learning Representations for Continual Learning

By Khurram Javed, Martha White. 2019

385 citations recorded by the source corpus.

A continual learning agent should be able to build on top of existing knowledge to learn on new data quickly while minimizing forgetting. Current intelligent systems based on neural network function approximators arguably do the opposite---they are highly prone to forgetting and rarely trained to facilitate future learning. One reason for this poor behavior is that they learn from a representation that is not explicitly trained for these two goals. In this paper, we propose OML, an objective that directly minimizes catastrophic interference by learning representations that accelerate future learning and are robust to forgetting under online updates in continual learning. We show that it is possible to learn naturally sparse representations that are more effective for online updating. Moreover, our algorithm is complementary to existing continual learning strategies, such as MER and GEM. Finally, we demonstrate that a basic online updating strategy on representations learned by OML is competitive with rehearsal based methods for continual learning. We release an implementation of our method at this https URL .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “AIM[183]learnsamixtureofexpertstomakepredic-tionswiththerepresentationsofOML[181]orANML[182], furthersparsifyingtherepresentationsatthearchitecturallevel.”
Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer (2022, methodology): “Meta-learning based methods [21, 26] are also useful while learning continually.”
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “Continual learning in practice The study of continual learning (or lifelong learning) dates back to the work of [34] and it receives a surge of research interest over recent years [14, 17, 10, 5, 13, 29, 32, 16, 27, 37].”
Memory Efficient Continual Learning with Transformers (2022, methodology): “…21]; (4) Parameter isolation-based methods [25, 62] allocate model parameters dedicated to different tasks and mask them out when learning a new task; (5) Meta-learning-based methods, which directly optimize the knowledge transfer among tasks [47, 37] or learn robust data representations [24, 60].”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “…2018), Diffusion-based Neuromodulation (Velez and Clune, 2017), IMM (Lee et al., 2017), Expandable Networks (Yoon et al., 2018; Li et al., 2019), RPSNet (Rajasegaran et al., 2019), reinforcement learning (Kaplanis et al., 2019; Rolnick et al., 2019), and meta-learning (Javed and White, 2019).”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “Several works propose meta-learning-based approaches for LL (Riemer et al., 2019; Finn et al., 2019; Javed & White, 2019; Wang et al., 2020).”
CoMPS: Continual Meta Policy Search (2021, background): “Adaptation without task boundaries or inside of an RL episode has also become a new area of investigation for meta-learning (Nagabandi et al., 2019; Javed & White, 2019; Harrison et al., 2019; Al-Shedivat et al., 2018; He et al., 2019; Harrison et al., 2019).”
Continual Learning via Local Module Composition (2021, background): “Continual-meta learning is often confused with its counterpart meta-continual learning [40, 11, 93], in which algorithm are learning to continually learn.”
Wide Neural Networks Forget Less Catastrophically (2021, background): “For example, Javed and White (2019); Beaulieu et al. (2020) showed that meta-learned representations forget less when trained continually.”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “Some works [69, 34] combine rehearsal with metalearning to find the balance between transfer and interference.”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “Javed and White [223] proposed a meta-objective to learn representations which are naturally highly sparse and thus effectively mitigate catastrophic forgetting in continual learning.”
Few-Shot and Continual Learning with Attentive Independent Mechanisms (2021, methodology): “We adopt the model from OML [22] and ANML [5] with a slight modification for our experiments.”
Co2L: Contrastive Continual Learning (2021, background): “Only a few recent studies on continual learning focus on representations in two aspects: how to maintain learned representations [33] and how to learn representations accelerating future learning [23, 17, 43, 15].”
Learngene: From Open-World to Your Learning Task (2021, background): “Although meta continual learning (Yao et al. 2020; Gidaris and Komodakis 2018; Jerfel et al. 2019; Javed and White 2019; Gupta, Yadav, and Paull 2020) can solve the sequential task problem by recording the efﬁciency task buffer, it still cannot prevent data-interaction.”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, methodology): “It has been used as a training strategy in sequence learning [43, 44] and continual learning scenarios [45, 46, 47, 48].”
Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021, background): “The field of continual learning has grown considerably in the past years, with numerous works forming new subfields [22] and finding novel applications [45].”
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, methodology): “, 2016; Mallya and Lazebnik, 2018); (iv) metalearning-based method, which learns robust model initialization (Obamuyide and Vlachos, 2019) or data representations (Javed and White, 2019; Holla et al., 2020; Wang et al., 2020) before training on task sequences to alleviate forgetting.”
Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “Recently, some attention has been devoted to latent regularization in continual image classification [1, 2, 27].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “OML [129] is a meta-objective that uses interference as a training signal to learn a representation that accelerates future learning and avoid catastrophic interference.”