Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

By Chelsea Finn, Sergey Levine. 2017

236 citations recorded by the source corpus.

Learning to learn is a powerful paradigm for enabling models to learn from data more effectively and efficiently. A popular approach to meta-learning is to train a recurrent model to read in a training dataset as input and output the parameters of a learned model, or output predictions for new test inputs. Alternatively, a more recent approach to meta-learning aims to acquire deep representations that can be effectively fine-tuned, via standard gradient descent, to new tasks. In this paper, we consider the meta-learning problem from the perspective of universality, formalizing the notion of learning algorithm approximation and comparing the expressive power of the aforementioned recurrent models to the more recent approaches that embed gradient descent into the meta-learner. In particular, we seek to answer the following question: does deep representation combined with standard gradient descent have sufficient capacity to approximate any learning algorithm? We find that this is indeed true, and further find, in our experiments, that gradient-based meta-learning consistently leads to learning strategies that generalize more widely compared to those represented by recurrent models.

Subfields: Artificial Intelligence, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

Meta-learning with differentiable closed-form solvers (2018, methodology): “A recent and promising research direction is the one set by Maclaurin et al. (2015) and by the MAML algorithm (Finn et al., 2017; Finn & Levine, 2018).”
Adaptive Gradient-Based Meta-Learning Methods (2019, methodology): “An expressivity result was shown for MAML by Finn and Levine [22], proving that the meta-learner can approximate any permutation-invariant learner given enough data and a specific neural architecture.”
Meta-Gradient Reinforcement Learning (2018, background): “e the reinforcement learning problem, so that the recurrent policy can generalise into new tasks faster than learning the policy from scratch. Model-Agnostic Meta-Learning (MAML) [Finn et al., 2017a, Finn and Levine, 2018, Finn et al., 2017b, Grant et al., 2018, Al-Shedivat et al., 2018] learns a go”
CoMPS: Continual Meta Policy Search (2021, methodology): “Because the adaptation process in MAML corresponds to a well-defined learning algorithm, even new out-ofdistribution tasks are learned (albeit slowly), while tasks that resemble those seen previously will be learned much more quickly (Finn & Levine, 2018).”
Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (2018, methodology): “MAML is an optimization based meta-learning algorithm with nice properties such as the ability to approximate any learning algorithm and the ability to generalize well to learning data outside of the previous distribution (Finn & Levine, 2017).”
Machine Theory of Mind (2018, neutral): “There should be nothing surprising about seeing the ToM-net learn to approximate Bayes-optimal online inference; this should be expected given more general results about inference and meta-learning with neural networks (MacKay, 1995; Finn & Levine, 2017).”
Meta-Learning: A Survey (2018, neutral): “Finn and Levine [52] also argue that MAML is able to”
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (2019, methodology): “This algorithm, with deep neural networks as the underlying model, has been highly influential, with significant follow on work [19, 6, 23, 11, 5, 9, 28, 31, 12].”
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning (2018, background): “Despite the update rule being ﬁxed, a learned initialization of an overparameterized deep network followed by gradient descent is as expressive as update rules represented by deep recurrent networks (Finn and Levine, 2017).”
Generalizing from a Few Examples (2019, background): “However, in streaming applications, p(T ) is dynamic [38] and new tasks are continually arriving.”
Meta-Reinforcement Learning of Structured Exploration Strategies (2018, methodology): “Our approach is most closely related to model-agnostic meta-learning (MAML) (Finn et al., 2017a), which directly trains for model parameters that can adapt quickly with standard gradient descent.”
Evolved Policy Gradients (2018, background): “Rather than mining rules that generalize across data points, as in traditional machine learning, metalearning concerns itself with devising algorithms that generalize across tasks, by infusing prior knowledge of the task distribution (12).”
Multi-agent deep reinforcement learning: a survey (2021, methodology): “This learning-to-learn approach is known as meta-learning (Finn and Levine 2018; Schmidhuber et al. 1996).”
Evolving Reinforcement Learning Algorithms (2021, background): “Finn et al. (2017); Finn & Levine (2018) meta-learns initial parameters which can quickly adapt to new tasks, while RL 2 (Duan et al., 2016) and concurrent work (Wang et al., 2017), formulates RL itself as a learning problem that is learned with an RNN.”