Alpha MAML: Adaptive Model-Agnostic Meta-Learning

By Harkirat Singh Behl, Atilim Gunes Baydin, Philip H. S. Torr. 2019

72 citations recorded by the source corpus.

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinforcement learning, but comes with the need for costly hyperparameter tuning for training stability. We address this shortcoming by introducing an extension to MAML, called Alpha MAML, to incorporate an online hyperparameter adaptation scheme that eliminates the need to tune meta-learning and learning rates. Our results with the Omniglot database demonstrate a substantial reduction in the need to tune MAML training hyperparameters and improvement to training stability with less sensitivity to hyperparameter choice.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Personalized Federated Learning: A Meta-Learning Approach (2020, background): “…was ﬁrst proposed by Finn et al. (2017) and followed by a number of papers studying its empirical characteristics (Antoniou et al., 2019; Li et al., 2017; Grant et al., 2018; Nichol et al., 2018; Zintgraf et al., 2019; Behl et al., 2019) as well as its convergence properties (Fallah et al., 2019).”
La-MAML: Look-ahead Meta Learning for Continual Learning (2020, background): “α-MAML [28] analytically updates the two scalar LRs in the MAML update for more adaptive few-shot learning.”
Personalized Federated Learning using Hypernetworks (2021, background): “Many papers proposed a decentralized version of the model agnostic meta-learning (MAML) problem (Fallah et al., 2020a; Li et al., 2017; Behl et al., 2019; Zhou et al., 2019; Fallah et al., 2020b).”
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms (2019, methodology): “Several papers have studied the empirical performance of MAML for nonconvex settings [Al-Shedivat et al., 2018; Antoniou et al., 2019; Behl et al., 2019; Grant et al., 2018; Li et al., 2017; Nichol et al., 2018; Zintgraf et al., 2019].”
When MAML Can Adapt Fast and How to Assist When It Cannot (2019, background): “, 2017) while Alpha MAML adapts those learning rates during fast adaptation (Behl et al., 2019).”