Semi-Implicit Variational Inference
By Mingzhang Yin, Mingyuan Zhou. 2018
150 citations recorded by the source corpus.
Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generated via reparameterization. Not only does SIVI expand the variational family to incorporate highly flexible variational distributions, including implicit ones that have no analytic density functions, but also sandwiches the evidence lower bound (ELBO) between a lower bound and an upper bound, and further derives an asymptotically exact surrogate ELBO that is amenable to optimization via stochastic gradient ascent. With a substantially expanded variational family and a novel optimization algorithm, SIVI is shown to closely match the accuracy of MCMC in inferring the posterior in a variety of Bayesian inference tasks.
Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- George Tucker
- Dmitry P. Vetrov
- Dmitry Molchanov
- Bo Dai
- Zhangyang Wang
- Max Welling
- Artem Sobolev
In Probabilistic & Bayesian Methods
- Dmitry P. Vetrov
- Artem Sobolev
- George Tucker
- Dieterich Lawson
- Dmitry Molchanov
- Valery Kharitonov
- Bo Dai
- Zhangyang Wang
Citations in the corpus
- Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, background): “As a way of increasing the flexibility of the variational family, Yin and Zhou [68] introduce the idea of semi-implicit variational families.”
- VariGrow: Variational Architecture Growing for Task-Agnostic Continual Learning based on Bayesian Novelty (2022, methodology): “Meanwhile, each component is handled by an expert distribution defined implicitly through Bayesian Neural Networks (BNNs) (Yin & Zhou, 2018; Titsias & Ruiz, 2019; Molchanov et al., 2019).”
- De Novo Protein Design for Novel Folds using Guided Conditional Wasserstein Generative Adversarial Networks (gcWGAN) (2019, background): “Notably, gcWGAN designs are much more diverse, which can be attributed to that WGAN with its implicit model and Wasserstein distance overcomes known limitations of VAE (Yin and Zhou, 2018; Arjovsky et al., 2017; Davidson et al., 2018).”
- Stochastic Normalizing Flows (2020, unknown): “Alternatively, we lie in the setting of semi-implicit variational inference seen in Yin & Zhou (2018) and Titsias & Ruiz (2019), and those techniques directly extend to our case as well.”
- Advances in Variational Inference (2017, unknown): “example a hierarchical structure [202], [217] .”
- Foundations and Modeling of Dynamic Networks Using Dynamic Graph Neural Networks: A Survey (2020, methodology): “By integrating semi-implicit variational inference [103] with VGRNN create SI-VGRNN.”
- Doubly Semi-Implicit Variational Inference (2018, methodology): “We can also apply the importance-weighted lower bound [4] similarly to the importance weighted SIVAE [40], and obtain IW-DSIVAE, a lower bound on the IWAE objective for a variational autoencoder with a semi-implicit prior and a semi-implicit posterior.”
- Importance Weighted Hierarchical Variational Inference (2019, methodology): “, 2016), Semi-Implicit Variational Inference (Yin and Zhou, 2018) and Doubly Semi-Implicit Variational Inference (Molchanov et al.”