Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation
By Justin Domke, D. Sheldon. 2019
21 citations recorded by the source corpus.
Recent work in variational inference (VI) uses ideas from Monte Carlo estimation to tighten the lower bounds on the log-likelihood that are used as objectives. However, there is no systematic understanding of how optimizing different objectives relates to approximating the posterior distribution. Developing such a connection is important if the ideas are to be applied to inference-i.e., applications that require an approximate posterior and not just an approximation of the log-likelihood. Given a VI objective defined by a Monte Carlo estimator of the likelihood, we use a "divide and couple" procedure to identify augmented proposal and target distributions. The divergence between these is equal to the gap between the VI objective and the log-likelihood. Thus, after maximizing the VI objective, the augmented variational distribution may be used to approximate the posterior distribution.
Subfields: Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Probabilistic & Bayesian Methods
- George Tucker
- Dieterich Lawson
- Bo Dai
- Rajesh Ranganath
- David M. Blei
- Dustin Tran
- Michalis K. Titsias
- Qiang Liu
Citations in the corpus
- Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, background): “Concurrently, Domke and Sheldon [19] note a similar connection, however, their focus is on the use of the variational distribution for posterior inference.”