Variational Cumulant Expansions for Intractable Distributions
By David Barber, Piërre van de Laar. 2011
15 citations recorded by the source corpus.
Intractable distributions present a common difficulty in inference within the probabilistic knowledge representation framework and variational methods have recently been popular in providing an approximate solution. In this article, we describe a perturbational approach in the form of a cumulant expansion which, to lowest order, recovers the standdard Kullback-Leibler variational bound. Higher-order terms describe corrections on the variational approach without incurring much further computational cost. The relationship to other perturbational approaches such as TAP is also elucidated. We demonstrate the method on a particular class of undirected graphical models, Boltzmann machines, for which our simulation results confirm improved accuracy and enhanced stability during learning.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- David M. Blei
- Alp Kucukelbir
- Jon D. McAuliffe
- Stephan Mandt
- Rajesh Ranganath
- David Barber
- Dustin Tran
- Yoshua Bengio
Citations in the corpus
- Variational Inference: A Review for Statisticians (2016, background): “Other work has developed divergences based on lower bounds that are tighter than the ELBO (Barber and de van Laar 1999; Leisink and Kappen 2001).”
- Tractable Variational Structures for Approximating Graphical Models (1998, background): “[12] D.”