Importance Weighted Hierarchical Variational Inference
By Artem Sobolev, Dmitry P. Vetrov. 2019
32 citations recorded by the source corpus.
Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by the requirement of having a tractable density function. To overcome this roadblock, we introduce a new family of variational upper bounds on a marginal log density in the case of hierarchical models (also known as latent variable models). We then give an upper bound on the Kullback-Leibler divergence and derive a family of increasingly tighter variational lower bounds on the otherwise intractable standard evidence lower bound for hierarchical variational distributions, enabling the use of more expressive approximate posteriors. We show that previously known methods, such as Hierarchical Variational Models, Semi-Implicit Variational Inference and Doubly Semi-Implicit Variational Inference can be seen as special cases of the proposed approach, and empirically demonstrate superior performance of the proposed method in a set of experiments.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Fei Ye
- Adrian G. Bors
- George Tucker
- Sergey Levine
- Bo Dai
- Rajesh Ranganath
- Dieterich Lawson
- Ofir Nachum
In Computer Vision and Pattern Recognition
- Fei Ye
- Adrian G. Bors
- Bo Dai
- George Tucker
- Dieterich Lawson
- Rajesh Ranganath
- Luc Van Gool
- Xiaogang Wang
In Probabilistic & Bayesian Methods
- George Tucker
- Dieterich Lawson
- Fei Ye
- Adrian G. Bors
- Bo Dai
- Rajesh Ranganath
- David M. Blei
- Dustin Tran
Citations in the corpus
- Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, background): “Due to the usefulness of such a bound, there has been an intense effort to provide improved bounds [9, 48, 51, 41, 68, 50, 63].”
- Continual Variational Autoencoder Learning via Online Cooperative Memorization (2022, background): “(2) can be used as the estimator for the data likelihood [50].”
- Lifelong Generative Modelling Using Dynamic Expansion Graph Model (2021, methodology): “2016) and hierarchical variational inference (Sobolev and Vetrov 2019) (See details in Appendix-F from SM(1)).”