On First-Order Meta-Learning Algorithms

By Alex Nichol, Joshua Achiam, John Schulman. arXiv.org, 2018

2,619 citations recorded by the source corpus.

This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training

View Paper: https://arxiv.org/abs/1803.02999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

Learning and Evaluating General Linguistic Intelligence (2019, background): “Standard meta learning methods (Finn et al., 2017; Nichol et al., 2018) require the distribution of tasks that we want the model to adapt to to be known in advance (i.”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, methodology): “Orcun Cetintas | AutoML • Reptile [3]: Proposes a new algorithm with only first-order derivatives • MAML++ [4]: Stabilizes MAML training and proposes improvements such as learning the learning rate • Meta-SGD [5]: Learns all components of a meta-optimizer (initialization, update direction and learni”
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “Other relevant threads of work include research on meta-learning to learn model hyper-parameters (Nichol et al., 2018; Baydin et al., 2017), model parameters (Ha et al., 2016a; Platanios et al., 2018) and models that learn new tasks with high sample ef-ﬁciency (Finn et al., 2017) without forgetting…”
Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, methodology): “To further simplify MAML, Reptile [32] removed re-initialization for each task, making it a more natural choice in certain settings.”
Feature Learning in Infinite-Width Neural Networks (2020, background): “Second Order MAML generally achieves performance slightly better than First Order MAML, but at the cost of significantly slower updates [32].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “On the other side of the spectrum, methods that optimize standard iterative learning algorithms, such as backpropagating through gradient descent (Finn et al., 2017; Nichol et al., 2018) or explicitly learning the learner’s update rule (Hochreiter et al.”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, methodology): “Reptile (Nichol et al., 2018) is another memory-efficient, first-order meta-learning algorithm that first makes multiple gradient updates in the inner loop, then directly uses θ − θ to approxi-”
Advances and Open Problems in Federated Learning (2019, background): “Most notably, the training phase of the popular Reptile algorithm [308] is closely related to Federated Averaging [289] — Reptile allows for a server learning rate and assumes all clients have the same amount of data, but is otherwise the same.”
How to train your MAML (2018, methodology): “The results of Reptile vary, in some cases exceeding MAML, and in others producing results inferior to MAML.”
On the Outsized Importance of Learning Rates in Local Update Methods (2020, methodology): “The core idea has inspired a number of extensions (Antoniou et al., 2019; Nichol et al., 2018; Rusu et al., 2019; Grant et al., 2018; Rajeswaran et al., 2019; Raghu et al., 2020), which broadly use a two-level optimization structure to perform meta-learning.”
Backdoor Attacks on Federated Meta-Learning (2020, methodology): “A12 and A13) episodes of additional local meta-training (Reptile) with mini-ImageNet before supervised fine-tuning or matching networks training.”
Personalized Federated Learning with Moreau Envelopes (2020, background): “[36] discovered that FedAvg can be interpreted as meta-learning and proposed combining FedAvg with Reptile [33] for FL personalization.”
Personalized Federated Learning: A Meta-Learning Approach (2020, methodology): “…was ﬁrst proposed by Finn et al. (2017) and followed by a number of papers studying its empirical characteristics (Antoniou et al., 2019; Li et al., 2017; Grant et al., 2018; Nichol et al., 2018; Zintgraf et al., 2019; Behl et al., 2019) as well as its convergence properties (Fallah et al., 2019).”
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization (2019, methodology): “To address this heterogeneity, other methods such as multitask learning and meta learning have been proposed to train multiple local models [Li et al., 2019b, Nichol et al., 2018, Smith et al., 2017].”
Improving Federated Learning Personalization via Model Agnostic Meta Learning (2019, methodology): “We note that this can be seen as an analogous observation to those of Nichol et al. (2018), where Reptile (1) yields a model with clearly inferior personalized performance.”
Differentially Private Meta-Learning (2019, methodology): “, 2017) and Reptile (Nichol et al., 2018), tries to learn a common initialization φ over a set of tasks t = 1, .”
LEAF: A Benchmark for Federated Settings (2018, methodology): “Our synthetic dataset (introduced in Section 2) is inspired by the one presented in [20], but has possible additional heterogeneity designed to make current meta-learning methods (such as Reptile [27]) fail.”
Adaptive Personalized Federated Learning (2020, background): “Jiang et al. (2019) discuss the similarity between federated learning and meta-learning approaches, notably the Reptile algorithm by Nichol et al. (2018) and FedAvg, and combine them to personalize local models.”