Meta-Learning with Implicit Gradients

By Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, Sergey Levine. 2019

1,013 citations recorded by the source corpus.

A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific models are learned in the inner-loop, by using only a small amount of data from the current task. A key challenge in scaling these approaches is the need to differentiate through the inner loop learning process, which can impose considerable computational and memory burdens. By drawing upon implicit differentiation, we develop the implicit MAML algorithm, which depends only on the solution to the inner level optimization and not the path taken by the inner loop optimizer. This effectively decouples the meta-gradient computation from the choice of inner loop optimizer. As a result, our approach is agnostic to the choice of inner loop optimizer and can gracefully handle many gradient steps without vanishing gradients or memory constraints. Theoretically, we prove that implicit MAML can compute accurate meta-gradients with a memory footprint that is, up to small constant factors, no more than that which is required to compute a single inner loop gradient and at no overall increase in the total computational cost. Experimentally, we show that these benefits of implicit MAML translate into empirical gains on few-shot image recognition benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Representation & Self-Supervised Learning

Citations in the corpus

Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation (2021, methodology): “In Rajeswaran et al. (2019), the regularization of the Hessian like we described in Section 3 appears due to regularizing L by adding λ 2 || Y − Y 0 || 2 .”
True Few-Shot Learning with Language Models (2021, methodology): “CV has also been used in prior work on multi-distribution few-shot learning [26, 27].”
On the Outsized Importance of Learning Rates in Local Update Methods (2020, methodology): “The core idea has inspired a number of extensions (Antoniou et al., 2019; Nichol et al., 2018; Rusu et al., 2019; Grant et al., 2018; Rajeswaran et al., 2019; Raghu et al., 2020), which broadly use a two-level optimization structure to perform meta-learning.”
Meta-Learning in Neural Networks: A Survey (2020, background): “Popular solutions include implicit differentiation of v [154], [163], [293], forward-mode differentiation of v [67], [69], [294], gradient preconditioning [105], evolutionary strategies [295], solving for a greedy version of v online by alternating inner and outer steps [21], [42], [198], truncation”
Meta-learning via Language Model In-context Tuning (2021, background): “, 2019) and nested optimization during meta-training is often unstable (Nichol et al., 2018; Antoniou et al., 2019; Rajeswaran et al., 2019).”
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2020, background): “Learning explanations can be modeled as a meta-learning problem, where the meta-objective is the few-shot test performance of the student trained with intermediate explanations, and this performance could serve as a signal to update the explanation generation module using implicit gradients as in (R”
How to Train Your MAML to Excel in Few-Shot Classification (2021, methodology): “Recent years have witnessed an abundance of new approaches on meta-learning [29, 30, 48, 61, 63, 64, 68, 77, 79, 80, 94, 98], developed in various areas including few-shot learning [70, 85, 89, 105, 107, 110, 111, 115], optimization [3, 9, 53, 108], reinforcement and imitation learning [23, 24, 31, ”
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (2021, methodology): “The gradient of the outer-loop for DARTS is then calculated as: DARTS considers the one-step unroll learning paradigm (Liu et al., 2019; Rajeswaran et al., 2019) for the hypergra-dient calculation.”
Seven Defining Features of Terahertz (THz) Wireless Systems: A Fellowship of Communication and Sensing (2021, background): “Furthermore, existing meta-learning techniques have been mostly procured on supervised learning problems [68]– [70], where data is inherently labeled.”
Boosting Few-Shot Learning With Adaptive Margin Loss (2020, neutral): “] ResNet12 Metric 59.00 77.46 AM3 (Prototypical Network) [35] ResNet12 Metric 65.21 0.49 75.20 0.36 MAML [7] 4Conv Gradient 48.70 1.84 63.11 0.92 MAML++ [1] 4Conv Gradient 52.15 0.26 68.32 0.44 iMAML [22] 4Conv Gradient 49.30 1.88 - LCC [19] 4Conv Gradient 54.6 0.4 71.1 0.4 CAML [11] ResNet12 Gradie”
Memory Efficient Meta-Learning with Large Images (2021, background): “First-order MAML [1] saves memory by avoiding the estimate of second-order derivatives.”
Multi-Step Model-Agnostic Meta-Learning: Convergence and Improved Algorithms (2020, methodology): “Among them, optimization-based meta-learning is a simple and effective approach used in a wide range of domains including classification/regression (Rajeswaran et al., 2019), reinforcement learning (Finn et al., 2017a), robotics (Al-Shedivat et al., 2017), federated learning (Chen et al., 2018),…”
Gradients are Not All You Need (2021, background): “…that we are trying to optimize is known and can be phrased in terms of the ﬁxed point of some dynamical system, there can often be signiﬁcant beneﬁt to computing derivatives implicitly rather than unrolling optimization explicitly Rajeswaran et al. [2019], Bai et al. [2019], Blondel et al. [2021].”
Offline Meta-Reinforcement Learning with Advantage Weighting (2020, methodology): “, 2017a), to satisfy both computational and memory constraints (Wu et al., 2018; Rajeswaran et al., 2019), and MACAW also uses a truncated optimization.”
A Sample Complexity Separation between Non-Convex and Convex Meta-Learning (2020, background): “…the optimization properties of gradient-based meta-learning algorithms have been recently studied in the non-convex setting [Fallah et al., 2019, Rajeswaran et al., 2019, Zhou et al., 2019], these results only provide stationary-point convergence guarantees and do not show a reduction in sample…”
Meta-learning without data via Wasserstein distributionally-robust model fusion (2022, neutral): “Most existing works [Vinyals et al., 2016, Finn et al., 2017, Snell et al., 2017, Ravi and Beatson, 2019, Rajeswaran et al., 2019, Finn et al., 2019b, Raghu et al., 2020, Wang et al., 2020b, Zhou et al., 2021, Bohdal et al., 2021, Rothfuss et al., 2021, CHEN et al., 2021, Bronskill et al., 2021,…”
Variable-Shot Adaptation for Online Meta-Learning (2020, methodology): “, 2017), and optimization-based meta-learners that embed an optimization procedure into the meta-learner (Finn et al., 2017; Li et al., 2017b; Rusu et al., 2018; Zintgraf et al., 2018; Bertinetto et al., 2018; Rajeswaran et al., 2019; Lee et al., 2019; Yoon et al., 2018; Finn et al., 2018).”
Optimizing Millions of Hyperparameters by Implicit Differentiation (2019, methodology): “We use a Neumann approximation for the inverse-Hessian, instead of CG [30, 9] or the identity.”
On Modulating the Gradient for Meta-learning (2020, background): “Gradient-based algorithms [3, 9, 31–33, 45] can be successfully employed be the total loss of the network where (cid:96) : X ×Y× Θ → R + denotes the loss for the input x i ∈ X and desired output y i ∈ Y .”