Meta-Learning with Warped Gradient Descent

By Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu, Hujun Yin, Raia Hadsell. 2019

221 citations recorded by the source corpus.

Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works have approached this issue either by attempting to train a neural network that directly produces updates or by attempting to learn better initialisations or scaling factors for a gradient-based update rule. Both of these approaches pose challenges. On one hand, directly producing an update forgoes a useful inductive bias and can easily lead to non-converging behaviour. On the other hand, approaches that try to control a gradient-based update rule typically resort to computing gradients through the learning process to obtain their meta-gradients, leading to methods that can not scale beyond few-shot task adaptation. In this work, we propose Warped Gradient Descent (WarpGrad), a method that intersects these approaches to mitigate their limitations. WarpGrad meta-learns an efficiently parameterised preconditioning matrix that facilitates gradient descent across the task distribution. Preconditioning arises by interleaving non-linear layers, referred to as warp-layers, between the layers of a task-learner. Warp-layers are meta-learned without backpropagating through the task training process in a manner similar to methods that learn to directly produce updates. WarpGrad is computationally efficient, easy to implement, and can scale to arbitrarily large meta-learning problems. We provide a geometrical interpretation of the approach and evaluate its effectiveness in a variety of settings, including few-shot, standard supervised, continual and reinforcement learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

Fast Model Editing at Scale (2021, methodology): “Various methods for meta-learning also use gradient transforms to achieve better model updates for few-shot learning (Ravi and Larochelle, 2017; Li et al., 2017; Lee and Choi, 2018; Park and Oliva, 2019; Flennerhag et al., 2020).”
Layerwise Optimization by Gradient Decomposition for Continual Learning (2021, neutral): “[12] proposed WarpGrad which layerwisely meta-learned to warp task loss surfaces across the joint task-parameter distribution to facilitate gradient descent.”
Embracing Change: Continual Learning in Deep Neural Networks. (2020, background): “Relevant to continual learning, outer loops can be de ﬁ ned that optimize for performance in non-stationary settings [76 – 78].”
Incremental Object Detection via Meta-Learning (2020, positive): “We intend to meta-learn a parameterised preconditioning matrix P (θ;φ), which warps the gradient to the steepest direction accounting for the different tasks [9], introduced to the object detector till then.”
Meta-Learning in Neural Networks: A Survey (2020, background): “Various meta-representations can be learned to improve continual learning performance, such as weight priors [134], gradient descent preconditioning matrices [105], or RNN learned optimizers [171], or feature representations [258].”
FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, background): “1Although there are more studies related to decoupling parameters (Rusu et al., 2018; Lee & Choi, 2018; Flennerhag et al., 2019; Chen et al., 2019b), we focus on decoupling the entire network into the body and head.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, background): “As far as enlarging parameter space, MT-NET [148] adds transformation matrix and a binary mask matrix; TAML [149] introduces regularisation conditions; OOD-MAML [150] draws on dummy samples; ARML [151] constructs meta-knowledge graphs; WGDMAML [152] places non-linear activation function between lear”
Learned Initializations for Optimizing Coordinate-Based Neural Representations (2020, methodology): “Most relevant to this work are optimization-based metalearning algorithms such as Model-Agnostic Meta Learning (MAML) [6] and Reptile [25], as well as various extensions [1, 5, 7, 17, 29].”
On Modulating the Gradient for Meta-learning (2020, background): “The general framework of preconditioning for fast adaptation of neural network is introduced in [11].”
Online Structured Meta-learning (2020, unknown): “Under this setting, there are two representative lines of meta-learning algorithm, including optimization-based meta-learning [7–9, 13, 11, 17, 19, 26, 28, 31] and non-parametric meta-learning [12, 22, 24, 34, 35, 38, 43].”
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks (2019, background): “They precondition the inner-gradients with diagonal or block-diagonal matrix (Flennerhag et al., 2020), which seems to provide extra flexibility to handle task and class imbalance.”
Bootstrapped Meta-Learning (2021, background): “The meta-objective is the validation loss in expectation over the task distribution: Several works have extended this setup by altering the update rule ( ϕ ) (Lee & Choi, 2018; Zintgraf et al., 2019; Park & Oliva, 2019; Flennerhag et al., 2020).”
DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning (2020, unknown): “, [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], target at effectively”
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks (2020, methodology): “Fast adaptation: Flennerhag et al. (2019) proposed an approach which mitigates slow adaption often observed in MAML by learning to warp the task loss surface to enable rapid descent to the loss minima.”
Introducing Symmetries to Black Box Meta Reinforcement Learning (2021, methodology): “…learning a weight initialization and adapting it with a human-engineered RL algo-rithm (Finn, Abbeel, and Levine 2017), warping computed gradients (Flennerhag et al. 2020), meta-learning hyper-parameters (Sutton 1992; Xu, van Hasselt, and Silver 2018) or meta-learning objective functions…”
Learning a Universal Template for Few-shot Dataset Generalization (2021, background): “Moreover, (Flennerhag et al., 2019) propose to interleave meta-learned ‘warp-layers’ be-tween the layers of a task learner, in the context of gradient-based meta-learning.”