Amortized Inference in Probabilistic Reasoning

By Samuel J. Gershman, Noah D. Goodman. 2014

393 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Variational Inference with Normalizing Flows (2015, methodology): “A second important practice is that the approximate posterior distribution q φ ( · ) is represented using a recognition model or inference network (Rezende et al., 2014; Dayan, 2000; Gershman & Goodman, 2014; Kingma & Welling, 2014).”
Generative Visual Manipulation on the Natural Image Manifold (2016, background): “In some sense the loss for one image provides information for many more images that share a similar appearance [29].”
Foundation Posteriors for Approximate Probabilistic Inference (2022, positive): “Amortized inference [26, 55] was proposed as a more efﬁcient alternative by learning a function q ( z x ) that maps a observed assignment x to a distribution over the latent variable z .”
An Introduction to Probabilistic Programming (2018, methodology): “This setting has been described as amortized inference (Gershman and Goodman, 2014), due to the tradeoff made between up-front and inference-time computation.”
Variational Autoencoders for Collaborative Filtering (2018, background): “Vaes make use of amortized inference [12]: they flexibly reuse inferences to answer related new problems.”
Lifelong Generative Modeling (2017, background): “L θ ( x ) ≤ P θ ( x ) (full derivation available in the appendix) By sharing the variational parameters φ of the encoder across the data points ( amortized inference Gershman & Goodman (2014)), variational autoencoders avoid per-data optimization loops typically needed by mean-ﬁeld approaches.”
Learning Disentangled Representations with Semi-Supervised Deep Generative Models (2017, background): “This approach allows us also amortise inference [7, 23, 29, 34], simultaneously learning a network that performs approximate inference over representations at the same time we learn the unknown factors of the model itself.”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, methodology): “…latent factors based on the popular and scalable framework of amortized variational inference (Kingma and Welling, 2013; Stuhlm¨uller et al., 2013; Gershman and Goodman, 2014; Rezende et al., 2014) powered by stochastic optimization (Hoﬀman et al., 2013; Kingma and Welling, 2013; Rezende et al.,…”
Automatic differentiation in machine learning: a survey (2015, methodology): “Amortized inference (Gershman and Goodman, 2014; Stuhlmüller et al., 2013) techniques based on deep learning (Le et al.”
Sylvester Normalizing Flows for Variational Inference (2018, neutral): “In continuous latent variable models, variational inference can be made particularly efﬁcient through the amortized inference, in which inference networks amortize the cost of calculating the variational posterior for a data point [Gershman and Goodman, 2014].”
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning (2016, methodology): “The idea of amortized inference (Gershman & Goodman, 2014) has been recently applied in various domains of probabilistic reasoning, including both amortized variational inference (e.g., Kingma & Welling, 2013; Rezende & Mohamed, 2015a), and data-driven proposals for (sequential) Monte Carlo methods…”
Seeing What a GAN Cannot Generate (2019, methodology): “The method does benefit from the power of a deep network to learn generalized rules [13], and the results are marginally better than the direct optimization of z.”
Iterative Amortized Inference (2018, methodology): “Optimization of each data example’s approximate posterior parameters, λ i ) , is replaced with the optimization of a shared, i.e. amortized (Gershman & Goodman, 2014), set of parameters, φ , contained within an inference model, f , of the form: While inference models have a long history, e.g.…”
Inference Compilation and Universal Probabilistic Programming (2016, background): “There has been a spate of recent work addressing the production of artifacts via “compiling away” or “amortizing” inference (Gershman and Goodman, 2014).”
Variational Rejection Sampling (2018, methodology): “To scale to large datasets, inference is amortized by introducing a recognition model approximating the true posterior over the latent variables, conditioned on the observed data [Dayan et al., 1995, Gershman and Goodman, 2014].”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, background): “Amortized variational inference (Gershman & Goodman, 2014; Ritchie et al., 2016) expands on this idea by using generative models to capture inference mappings.”