Recasting Gradient-Based Meta-Learning as Hierarchical Bayes

By Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, T. Griffiths. 2018

553 citations recorded by the source corpus.

Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm's operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning (2021, methodology): “Third, optimization-based approaches search for good initial parameters during training and adapt the pretrained model for new tasks (Finn et al., 2017; Lee & Choi, 2018; Grant et al., 2018).”
Meta-Learning Probabilistic Inference for Prediction (2018, methodology): “In Section 4, we show that ML-PIP re-frames and extends existing point-estimate probabilistic interpretations of meta-learning (Grant et al., 2018; Finn et al., 2018) to cover a broader class of methods, including gradient based meta-learning (Finn et al.”
Meta-Learning with Latent Embedding Optimization (2018, background): “Probabilistic meta-learning approaches such as those of Bauer et al. (2017) and Grant et al. (2018) have shown the advantages of learning Gaussian posteriors over model parameters.”
On the Outsized Importance of Learning Rates in Local Update Methods (2020, methodology): “The core idea has inspired a number of extensions (Antoniou et al., 2019; Nichol et al., 2018; Rusu et al., 2019; Grant et al., 2018; Rajeswaran et al., 2019; Raghu et al., 2020), which broadly use a two-level optimization structure to perform meta-learning.”
Personalized Federated Learning: A Meta-Learning Approach (2020, background): “…was ﬁrst proposed by Finn et al. (2017) and followed by a number of papers studying its empirical characteristics (Antoniou et al., 2019; Li et al., 2017; Grant et al., 2018; Nichol et al., 2018; Zintgraf et al., 2019; Behl et al., 2019) as well as its convergence properties (Fallah et al., 2019).”
Meta-Gradient Reinforcement Learning (2018, positive): “Model-Agnostic Meta-Learning (MAML) [Finn et al., 2017a, Finn and Levine, 2018, Finn et al., 2017b, Grant et al., 2018, Al-Shedivat et al., 2018] learns a good initialisation of the model that can adapt quickly to other tasks within a few gradient update steps.”
Task Agnostic Meta-Learning for Few-Shot Learning (2018, neutral): “LLAMA [23] 49.”
Causal Reasoning from Meta-reinforcement Learning (2019, background): “Analogous (Grant et al., 2018) models that learn causal structure directly from the environment, rather than having a pre-conceived formal theory, have also been implicated in human intelligence (Goodman et al., 2011).”
Probabilistic Model-Agnostic Meta-Learning (2018, methodology): “Other work that considers model uncertainty in the few-shot learning setting is the LLAMA method [15], which also builds on the MAML algorithm.”
Bayesian Model-Agnostic Meta-Learning (2018, methodology): “This probabilistic interpretation can be extended further to applying empirical Bayes to a hierarchical probabilistic model (Grant et al., 2018) as follows:”
Continual Learning via Local Module Composition (2021, methodology): “The meta-learner then leverages the connection between gradient-based meta-learning and hierarchical Bayes [29] for selecting the most likely expert for the given task at hand.”
Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting (2018, methodology): “[34] and [9] have independently proposed the use of block-diagonal Kronecker factored curvature approximations [23, 2] to sample from an approximate Gaussian posterior over the weights of a neural network.”
Neural Processes (2018, background): “Interestingly, recent work by Grant et al. (2018) out the relation between model agnostic meta learning (MAML, Finn et al. (2017)) and hierarchical Bayesian inference.”
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (2019, methodology): “This algorithm, with deep neural networks as the underlying model, has been highly influential, with significant follow on work [19, 6, 23, 11, 5, 9, 28, 31, 12].”
Automatically Composing Representation Transformations as a Means for Generalization (2018, background): “Much recent met-alearning work [6, 26, 27, 31, 37, 38, 53, 56, 60, 64, 79] focuses on domains where achieving zero-shot generalization is unclear and follow a common paradigm which assumes that the shared representation of monolithic architectures (with the exception of concurrent work [2]) can be…”