Bayesian parameter estimation via variational methods

By Tommi S. Jaakkola, Michael I. Jordan. 2000

658 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “An early example is Jaakkola and Jordan (1997, 2000), who developed a variational bound tailored to logistic regression.”
Weight Uncertainty in Neural Networks (2015, background): “…θ q ( w | θ ) log q ( w | θ ) P ( w ) P ( D| w ) d w The resulting cost function is variously known as the variational free energy (Neal and Hinton, 1998; Yedidia et al., 2000; Friston et al., 2007) or the expected lower bound (Saul et al., 1996; Neal and Hinton, 1998; Jaakkola and Jordan, 2000).”
Fast Adaptation in Generative Models with Generative Matching Networks (2016, methodology): “Hence, the resulting approximate predictive distribution and the corresponding estimate is aligned with practical usage of the model and is often considered in the literature (Snelson & Ghahramani, 2005; Jaakkola & Jordan, 2000).”
Variational MCMC (2001, methodology): “However, if the bounds on the likelihood of the observed and omplete data are hosen arefully, some existing empiri al results suggest that this framework an performvery well in omplex s enarios (de Freitas, Niranjan and Gee 2000, Jaakkola and Jordan1999, Jaakkola and Jordan 2000).”
BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery (2021, methodology): “Approaches to variational inference with multiple sets of latent variables often use a mean-field factorization [24], in our case corresponding to qφ(P,L,Σ) = qφ(P )qφ(L)qφ(Σ).”
Variational inference for count response semiparametric regression (2013, neutral): “Jaakkola & Jordan (2000) derived a lower bound on the Bayesian logistic regression marginal likelihood that leads to tractable approximate ar X iv :1 30 9.”
Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations (2012, background): “One could construct a bound on log(1 + e x ) such as the “quadratic” bound (Jaakkola and Jordan (2000)) or the “tilted” bound (Saul and Jordan (1998)).”
Nonparametric variational inference (2012, methodology): “First, we show that it performs as well in Bayesian logistic regression as the method of Jaakkola & Jordan (2000), which is tailored to that speciﬁc model.”
Model-Based Clustering (2016, methodology): “Tang et al. (2015) propose using a variational 357 P.D. McNicholas approximation to the likelihood, as proposed by Jaakkola and Jordan (2000), Tipping (1999), and Attias (2000).”
Stochastic variational learning in recurrent spiking networks (2014, methodology): “The variational approach consists of approximating a complex distribution p by a simpler distribution q and provides a ﬂexible generalization of the expectation-maximization (EM) algorithm (see Jaakkola and Jordan, 2000; Beal and Ghahramani, 2006).”
Bounded Approximations for Marginal Likelihoods (2010, methodology): “…by machine learning and Bayesian learning communities for deterministic approximation of marginal distributions (MacKay, 1995; Jordan et al., 1999; Jaakkola and Jordan, 2000; Humphreys and Titterington, 2000; Ueda and Ghahramani, 2002; Jordan, 2004; Wang and Titterington, 2004), have been…”
Weight Uncertainty in Neural Network (2015, background): “, 2007) or the expected lower bound (Saul et al., 1996; Neal and Hinton, 1998; Jaakkola and Jordan, 2000).”
Multi‐Omics Factor Analysis—a framework for unsupervised integration of multi‐omics data sets (2018, methodology): “To enable an efficient inference for non-Gaussian likelihoods, we employ variational lower bounds on the likelihood (Jaakkola & Jordan, 2000; Seeger & Bouchard, 2012; see Appendix Supplementary Methods, Section 4).”
Bayesian reasoning and machine learning (2012, neutral): “In this particular case the benign nature of the posterior (the log posterior is concave, see below) suggests that a simple Laplace approximation may suffice (see [150] for a variational approximation).”
Weighted Low-Rank Approximations (2003, methodology): “In order to provide for a more robust method, we use the following variational bound on the logistic (Jaakkola & Jordan, 2000): log g(yx) ≥ log g(y˜x) + yx−y˜x 2 − tanh(˜ x/2) 4˜x x 2 − ˜ x 2 = − 1 4 tanh(˜ x/2) ˜ x x − y ˜ x tanh(˜ x/2) + Const, yielding the corresponding bound on the likelihood:…”
Weakly supervised learning via statistical sufficiency (2016, methodology): “Factorization of logistic loss has been derived in other contexts in literature [Jaakkola and Jordan, 2000; Gao et al., 2016].”
Two problems with variational expectation maximisation for time-series models (2011, methodology): “2Other variational bounds may also be used in learning (e.g., Jaakkola and Jordan 2000 ).”
Sparse Variational Analysis of Linear Mixed Models for Large Data Sets. (2011, methodology): “This article proposes a method for obtaining sparse approximate Bayes inferences in such problems using variational methods (Jordan et al., 1999; Jaakkola and Jordan, 2000).”