Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributions
By Michalis K. Titsias. 2017
23 citations recorded by the source corpus.
We introduce a new algorithm for approximate inference that combines reparametrization, Markov chain Monte Carlo and variational methods. We construct a very flexible implicit variational distribution synthesized by an arbitrary Markov chain Monte Carlo operation and a deterministic transformation that can be optimized using the reparametrization trick. Unlike current methods for implicit variational inference, our method avoids the computation of log density ratios and therefore it is easily applicable to arbitrary continuous and differentiable models. We demonstrate the proposed algorithm for fitting banana-shaped distributions and for training variational autoencoders.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Andrew Gordon Wilson
- Pavel Izmailov
- Dmitry P. Vetrov
- Wesley J. Maddox
- Timur Garipov
- Polina Kirichenko
- Yibo Yang
- Stephan Mandt
In Computer Vision and Pattern Recognition
- Andrew Gordon Wilson
- Pavel Izmailov
- Dmitry P. Vetrov
- Polina Kirichenko
- Wesley J. Maddox
- Timur Garipov
- Paris Perdikaris
- Yibo Yang
Citations in the corpus
- Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems (2019, background): “Overall, we must underline that such variational inference techniques are trading the rigorous asymptotic convergence guarantees that sampling-based methods like Markov Chain Monte Carlo enjoy, in favor of enhanced computational efficiency and performance, although new unifying ideas are aiming to b”
- Subspace Inference for Bayesian Deep Learning (2019, background): “Alternatively, both Titsias (2017) and Krueger et al. (2017) propose Bayesian versions of hyper-networks to store meta-models of parameters.”
- Advances in Variational Inference (2017, background): “Implicit distributions for VI is an active area of research [72], [81], [102], [105], [107], [119], [129], [196], [210].”