Learning to learn by gradient descent by gradient descent

By Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, Nando de Freitas. 2016

2,222 citations recorded by the source corpus.

The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way. Our learned algorithms, implemented by LSTMs, outperform generic, hand-designed competitors on the tasks for which they are trained, and also generalize well to new tasks with similar structure. We demonstrate this on a number of tasks, including simple convex problems, training neural networks, and styling images with neural art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Optimization & Training

Citations in the corpus

Language Models are Few-Shot Learners (2020, methodology): “The StoryCloze 2016 dataset [MCH + 16] involves selecting the correct ending sentence for ﬁve-sentence long stories.”
Meta-Learning for Low-Resource Neural Machine Translation (2018, background): “There are two categories of meta-learning: 1. learning a meta-policy for updating model parameters (see, e.g., Andrychowicz et al., 2016; Ha et al., 2016a; Mishra et al., 2017) 2. learning a good parameter initialization for fast adaptation (see, e.g., Finn et al., 2017; Vinyals et al., 2016; Snell…”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “…how to learn rules for learning [Schmidhuber, 1987, Bengio et al., 1992], train meta-agents that control parameter updates [Hochreiter et al., 2001, Andrychowicz et al., 2016], augment models with special memory mechanisms [Santoro et al., 2016, Schmidhuber, 1992], and maximize the degree to which…”
Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (2017, background): “In the context of modern deep networks, Andrychowicz et al. (2016) learned a gradient descent optimizer by gradient descent, itself parameterized as a recurrent network.”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, methodology): “The key idea underlying our method is to train the model’s initial parameters such that the model has maximal performance on a new task after the parameters have been updated through one or more gradient steps computed with a small amount of data from that new task.”
Learning to Optimize Neural Nets (2017, methodology): “…nets and show that it outperforms popular hand-engineered optimization algorithms like ADAM (Kingma & Ba, 2014), AdaGrad (Duchi et al., 2011) and RMSprop (Tieleman & Hinton, 2012) and an optimization algorithm learned using the supervised learning method proposed in (Andrychowicz et al., 2016).”
HyperNetworks (2016, background): “The concept of a network interacting with another network is central to the work of (Jaderberg et al., 2016; Andrychowicz et al., 2016), and especially (Denil et al., 2013; Yang et al., 2015; Bertinetto et al., 2016; De Brabandere et al., 2016), where certain parameters in a convolutional network…”
What can the brain teach us about building artificial intelligence? (2016, background): “Neural networks can also learn-to-learn by optimizing hyper-parameters, including the form of their weight update rule (Andrychowicz et al. 2016), over a set of related tasks. Although transfer learning and multitask learning are already important themes across AI, and in deep learning in particular”
Unrolled Generative Adversarial Networks (2016, background): “More recently, (Belanger & McCallum, 2015; Han et al., 2016; Andrychowicz et al., 2016) backpropagate through optimization procedures in contexts unrelated to GANs or minimax games.”
One-Shot Imitation Learning (2017, background): “Meta-learning has also been studied to discover neural network weight optimization algorithms [8, 9, 23, 50, 2, 31].”
Risks from Learned Optimization in Advanced Machine Learning Systems (2019, methodology): “However, research such as that in Andrychowicz et al.[4] and Duan et al.[5] demonstrate that optimization algorithms can be learned by RNNs, making it possible that a Universal Planning Networks-like agent could be entirely learned—assuming a very expressive model space—including the internal optimi”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “The meta-learner either learns to produce new parameters directly from the new data [9, 33, 56, 62, 64, 72, 73], or learns to produce an update rule to iteratively optimize the base learner to fit the new data [2, 6, 8, 38, 63, 97].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “A modern take on this idea has been presented by Andrychowicz et al. (2016) and Ravi & Larochelle (2017), showing beneﬁts on large-scale classiﬁcation, style transfer and few-shot learning.”
Rewriting a Deep Generative Model (2020, background): “Examples include domain adaptation [66], zero-shot or few-shot learning [71,51], model pre-training and feature learning [17,82,78], and meta-learning [8,4,23].”
Learning to Learn with Generative Models of Neural Network Checkpoints (2022, background): “They are typically trained with unrolled optimization (Hochreiter et al., 2001; Younger et al., 2001; Gregor & LeCun, 2010; Andrychowicz et al., 2016; Ravi & Larochelle, 2017; Wichrowska et al., 2017; Lv et al., 2017; Metz et al., 2019; 2022) or reinforcement learning (Li & Malik, 2016; 2017).”
Neural Architecture Search with Reinforcement Learning (2016, background): “More closely related is the idea of using a neural network to learn the gradient descent updates for another network (Andrychowicz et al., 2016) and the idea of using reinforcement learning to find update policies for another network (Li & Malik, 2016).”
Learning Transferable Architectures for Scalable Image Recognition (2017, background): “The concept of having one neural network interact with a second neural network to aid the learning process, or learning to learn or meta-learning [23, 49] has attracted much attention in recent years [1, 62, 14, 19, 35, 45, 15]. Most of these approaches have not been scaled to large problems like Im”
How to train your MAML (2018, background): “Recent work in meta-learning has produced state of the art results in a variety of settings (Wang et al., 2016; Ba et al., 2016; Zoph & Le, 2016; Andrychowicz et al., 2016; Vinyals et al., 2016; Li & Malik, 2016; Li et al., 2017; Antoniou et al 2017; Brock et al., 2017; Munkhdalai & Yu, 2017).”