Boosting Variational Inference: an Optimization Perspective

By Francesco Locatello, Rajiv Khanna, Joydeep Ghosh, Gunnar Rätsch. 2017

40 citations recorded by the source corpus.

Variational inference is a popular technique to approximate a possibly intractable Bayesian posterior with a more tractable one. Recently, boosting variational inference has been proposed as a new paradigm to approximate the posterior by a mixture of densities by greedily adding components to the mixture. However, as is the case with many other variational inference algorithms, its theoretical properties have not been studied. In the present work, we study the convergence properties of this approach from a modern optimization viewpoint by establishing connections to the classic Frank-Wolfe algorithm. Our analyses yields novel theoretical insights regarding the sufficient conditions for convergence, explicit rates, and algorithmic simplifications. Since a lot of focus in previous works for variational inference has been on tractability, our work is especially important as a much needed attempt to bridge the gap between probabilistic models and their corresponding theoretical properties.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

BooVAE: Boosting Approach for Continual Learning of VAE (2019, neutral): “Egorov et al. (2019), Locatello et al. (2018) used entropy-based regularization.”
MaxEntropy Pursuit Variational Inference (2019, methodology): “(1) is highly non-linear, we propose to follow the framework based on the Frank-Wolfe algorithm [26,17] and consider the constraint as a functional perturbation.”
Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective (2021, background): “Locatello et al. (2017, 2018) study convergence rates for a boosting based algorithm for iteratively reﬁned variational inference.”