Improving the Mean Field Approximation Via the Use of Mixture Distributions

By Tommi S. Jaakkola, Michael I. Jordan. 1999

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Variational Inference with Normalizing Flows (2015, background): “Another potentially powerful alternative would be to specify the approximate posterior as a mixture model, such as those developed by Jaakkola & Jordan (1998); Jordan et al. (1999); Gershman et al. (2012).”
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors (2020, methodology): “In terms of the speciﬁc distribution, instead of normalizing ﬂows we focus on mixtures, a well-known approach for expressive variational inference (Jaakkola & Jordan, 1998; Lawrence, 2001).”
Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning (2015, background): “| | Other lower bounds for the mutual information are also possible: Jaakkola & Jordan [9] present a lower bound by using the convexity bound for the logarithm; Brunel & Nadal [2] use a Gaussian assumption and appeal to the Cramer-Rao lower bound.”
Hierarchical Variational Models (2015, background): “, 2015) O(d(2)) O(d(2)) 3 discrete/continuous MIXTURE (Jaakkola and Jordan, 1998) O(Kd) O(Kd) 3 discrete/continuous NF (Rezende and Mohamed, 2015) O(Kd) O(Kd) 3 continuous-diff.”
DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data (2017, background): “For example, Ger-shman et al. [7], Jordan et al. [11] and Jaakkola et al. [10] model the approximate posterior of the inferred latent distribution as a mixture model to represent more complicated distributions.”
Semi-Implicit Variational Inference (2018, unknown): “While there exists a variety of methods that add some structure back to Q to partially restore the dependencies (Saul & Jordan, 1996; Jaakkola & Jordan, 1998; Hoffman & Blei, 2015; Giordano et al., 2015; Tran et al., 2015; 2016; Han et al., 2016; Ranganath et al., 2016; Maaløe et al., 2016; Gregor…”
Provable Bayesian Inference via Particle Mirror Descent (2015, background): “However, directly solving the optimization will lead to both computational and representational issues: one scan over the entire dataset at each iteration is needed, and the exact function update has no closed-form.”
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, methodology): “Proofs and related lemmas are in Appendix A.”
Variational Cumulant Expansions for Intractable Distributions (2011, background): “More (cid:13)exible approximating distributions have also been considered in the variational approach, for example, mixtures of factorised distributions (Jaakkola & Jordan, 1998; Lawrence et al., 1998), and decimatable structures (Saul & Jordan, 1994; Barber & Wiegerinck, 1999).”
Two problems with variational expectation maximisation for time-series models (2011, background): “mixtures of variational approximations Jaakkola and Jordan, 1998 ).”
Variational Boosting: Iteratively Refining Posterior Approximations (2016, methodology): “The component distributions can be any distribution over x from which we can draw samples using a continuous mapping that depends on λc (e.g., multivariate normals [16], or a composition of invertible maps [31]).”
The IM algorithm: a variational approach to Information Maximization (2003, background): “Certainly other well known lower bounds on the MI may be considered [6] and a future comparison of these different approaches would be interesting.”
Graphical Models, Exponential Families, and Variational Inference (2008, background): “Jaakkola and Jordan [118] explored the use of mixture distributions in improving the mean field approximation.”