Online Meta-Learning

By Chelsea Finn, Aravind Rajeswaran, Sham M. Kakade, Sergey Levine. 2019

325 citations recorded by the source corpus.

A central capability of intelligent systems is the ability to continuously build upon previous experiences to speed up and enhance learning of new tasks. Two distinct research paradigms have studied this question. Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch. In contrast, online (regret based) learning considers a sequential setting in which problems are revealed one after the other, but conventionally train only a single model without any task-specific adaptation. This work introduces an online meta-learning setting, which merges ideas from both the aforementioned paradigms to better capture the spirit and practice of continual lifelong learning. We propose the follow the meta leader algorithm which extends the MAML algorithm to this setting. Theoretically, this work provides an $\mathcal{O}(\log T)$ regret guarantee with only one additional higher order smoothness assumption in comparison to the standard online setting. Our experimental evaluation on three different large-scale tasks suggest that the proposed algorithm significantly outperforms alternatives based on traditional online learning approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Adaptive Gradient-Based Meta-Learning Methods (2019, neutral): “• Adapting to Dynamic Environments: While previous theoretical work has largely considered a fixed initialization [24, 34], in many practical applications of GBML the optimal initialization varies over time due to a changing environment [1].”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, neutral): “Several works propose meta-learning-based approaches for LL (Riemer et al., 2019; Finn et al., 2019; Javed & White, 2019; Wang et al., 2020).”
Efficient Continual Learning with Modular Networks and Task-Driven Priors (2020, neutral): “The CTrL (Continual Transfer Learning) benchmark is a collection of streams of tasks built over seven popular computer vision datasets, namely: CIFAR10 and CIFAR100 (Krizhevsky, 2009), DTD (Cimpoi et al., 2014), SVHN (Netzer et al., 2011), MNIST (LeCun et al., 1998), Rainbow-MNIST (Finn et al., 2019”
Meta-Consolidation for Continual Learning (2020, neutral): “Even meta-learning based continual learning methods that have been proposed in the recent past [34, 24, 6, 64], meta-learn an initialization amenable for quick adaptation across tasks, similar to MAML [23], and hence operate in the weight space.”
Continual Reinforcement Learning in 3D Non-stationary Environments (2019, neutral): “Recent works in this research area include meta-learning [12, 35, 1], hierarchical learning [50, 49] and continual learning approaches [21, 43, 37].”
Task Agnostic Continual Learning via Meta Learning (2019, neutral): “Finn et al. (2019) proposed an online meta-learning algorithm called follow the meta leader (FTML) based on the framework of regret-minimization.”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, positive): “FTML (Finn et al., 2019) extends the MAML algorithm to the online learning setting by incorporating the follow the leader (FTL) algorithm (Hannan, 1957).”
Continuous Meta-Learning without Tasks (2019, positive): “In the classiﬁcation setting, we apply MOCA to the Rainbow MNIST dataset of Finn et al. (2019).”
Towards mental time travel: a hierarchical memory for reinforcement learning agents (2021, neutral): “The agents can even extrapolate far beyond the training distribution to recall words after subsequent learning episodes without ever being trained to do so—better memory systems can help with the challenge of bridging from meta-learning to continual learning, which is receiving increasing interest [”
Online Structured Meta-learning (2020, unknown): “Metric NT FT FTML [10] DPM [15] HSML [41] OSML Acc.”
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning (2020, unknown): “FTML [17] extends the MAML algorithm to the online learning setting by incorporating the follow the leader (FTL) algorithm [23].”
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms (2019, unknown): “(12) Note that this assumption is less strict comparing to the bounded gradient assumption in [Finn et al., 2019; Rajeswaran et al., 2019].”
Learning to Interactively Learn and Assist (2019, unknown): “Further research should seek to minimize these limitations, perhaps through advances in online meta-learning (Finn et al. 2019).”
Reset-Free Lifelong Learning with Skill-Space Planning (2020, unknown): “Other works (Nagabandi et al., 2018a; Finn et al., 2019; Rolnick et al., 2018) are typically nonstationary at the abstraction of episodes, which generally does not require competencies such as online safe adaptation, and instead try to adapt quickly, minimize regret, or avoid catastrophic…”
A Collaborative Learning Framework via Federated Meta-Learning (2020, unknown): “Convexity and Smoothness of the Federated Meta-Learning Objective Function: Based on Theorem 1 in [17], we ﬁrst have the following result about the structural properties of function G ( θ ) .”
A Probabilistic Framework for Modular Continual Learning (2023, unknown): “They define a number of sequences, based on seven image classfication tasks, namely: CIFAR10 and CIFAR100 [19], DTD [3], SVHN [28], MNIST [20], RainbowMNIST [11], and Fashion MNIST [45].”
Self-Net: Lifelong Learning via Continual Self-Modeling (2018, unknown): “These techniques include Continual Learning with HyperNetworks (von Oswald et al., 2019), Task Agnostic Continual Learning via Meta-Learning (He et al., 2019), Online Meta-Learning (Finn et al., 2019), and Deep Online Meta-Learning (Nagabandi et al., 2018).”
Efficient Meta Learning via Minibatch Proximal Update (2019, unknown): “For lifelong learning, an follow-the-meta-leader extension of MAML has been studied in the setting of online learning [23].”