Meta-Amortized Variational Inference and Learning

By Kristy Choi, Mike Wu, Noah D. Goodman, Stefano Ermon. 2019

40 citations recorded by the source corpus.

Despite the recent success in probabilistic modeling and their applications, generative models trained using traditional inference techniques struggle to adapt to new distributions, even when the target distribution may be closely related to the ones seen during training. In this work, we present a doubly-amortized variational inference procedure as a way to address this challenge. By sharing computation across not only a set of query inputs, but also a set of different, related probabilistic models, we learn transferable latent representations that generalize across several related distributions. In particular, given a set of distributions over images, we find the learned representations to transfer to different data transformations. We empirically demonstrate the effectiveness of our method by introducing the MetaVAE, and show that it significantly outperforms baselines on downstream image classification tasks on MNIST (10-50%) and NORB (10-35%).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “Meta-Inference Meta-learning has been used to do inference amortized over both queries and a family of generative models [16, 30, 36].”
Meta-Learning an Inference Algorithm for Probabilistic Programs (2021, background): “…also features in the work on stochastic variational inference where a variational distribution receives information about each inference task in the form of its dataset of observations and is trained with a collection of datasets (Wu et al., 2020; Gordon et al., 2019; Iakovleva et al., 2020).”
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning (2019, methodology): “, 2016; Duan et al., 2016), building on work by Hochreiter et al. (2001) for supervised learning problems. At every time step, the network gets an auxiliary input indicating the action and reward of the preceding step. This allows learning within a task to happen online, entirely in the dynamics of ”