A Variational Approach to Bayesian Logistic Regression Models and their Extensions

By Tommi S. Jaakkola, Michael I. Jordan. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “This article, along with insights from statistical mechanics (Parisi 1988), led to a flurry of variational inference procedures for a wide class of models (Saul, Jaakkola, and Jordan 1996; Jaakkola and Jordan 1996, 1997; Ghahramani and Jordan 1997; Jordan et al. 1999).”
Bayesian Neural Word Embedding (2016, methodology): “We start by lower bounding ( | , ) p D U V using the following logistic bound [13] 2 2 log ( ) a σ where .”
Uncertainty-guided Continual Learning with Bayesian Neural Networks (2019, background): “Alternatively, variational inference provides a faster solution to the same problem in which the posterior is approximated using optimization rather than being sampled from a chain [30, 11, 13, 12].”
Minimum Probability Flow Learning (2009, background): “Of particular interest are contrastive divergence (CD), developed by Hinton, Welling and Carreira-Perpi˜n´an [24, 4], Hyv¨arinen’s score matching (SM) [7] and the minimum velocity learning framework proposed by Movellan [15, 14, 16].”
Iterative Splits of Quadratic Bounds for Scalable Binary Tensor Factorization (2014, methodology): “It is based on the partitioning of the tensor into blocks and the use of Jaakkola’s quadratic upper bound to the logistic loss [10].”
Non-conjugate Variational Message Passing for Multinomial and Binary Regression (2011, background): “[9] propose the “quadratic” bound on the integrand”
Piecewise Bounds for Estimating Bernoulli-Logistic Latent Gaussian Models (2011, methodology): “The variational quadratic bound introduced by Jaakkola & Jordan (1996) can be derived through Fenchel duality and has been quite widely used.”
Variational Message Passing (2005, methodology): “We take the approach of Jaakkola and Jordan (1996) and use a variational bound for the logistic sigmoid function defined as”
Gaussian Kullback-Leibler approximate inference (2013, background): “Another well studied route to obtaining a lower-bound for problems of the form of Equation (2) are so called local variational bounding procedures, see for example: Jaakkola and Jordan (1997), Gibbs and MacKay (2000), Girolami (2001), Palmer et al.”
Learning Multiple Related Tasks using Latent Independent Component Analysis (2005, methodology): “Solve the following Bayesian logistic regression (or other Bayesian classifier):q( ) argmaxq( ) (Z q( ) log N ( ; E [s℄; )QNi=1 p(yij ;xi)q( ) d ) 3.”
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient (2018, background): “Gaussian variational distributions with full covariance matrices have been used extensively for shallow models [6, 8, 16, 22, 24, 30, 34, 35].”
Learning from measurements in exponential families (2009, unknown): “(8) One could imagine using a normal approximation of q(θ) (Jaakkola and Jordan (1997); Seeger and Nickisch (2008)), but we chose a degenerate one for tractability.”
Black Box Variational Inference (2013, methodology): “In these settings, practitioners have resorted to model-speciﬁc algorithms (Jaakkola and Jordan, 1996; Blei and Laf-ferty, 2007; Braun and McAuliﬀe, 2007) or generic algorithms that require model speciﬁc computations (Knowles and Minka, 2011; Wang and Blei, 2013; Pais-ley et al., 2012).”
Latent Coincidence Analysis: A Hidden Variable Model for Distance Metric Learning (2012, methodology): “To handle larger data sets, we plan to explore online strategies for distance metric learning [18], possibly based on Bayesian [19] or conﬁdence-weighted updates [20].”
Modeling, Inference and Optimization With Composable Differentiable Procedures (2016, background): “The usual reference for variational inference in machine learning is Wainwright & Jordan [109].”
Firefly Monte Carlo: Exact MCMC with Subsets of Data (2014, background): “The logistic function has a family of scaled Gaussian lower bounds, described in Jaakkola and Jordan [1997], parameterized by ξ, the location at which the bound is tight:”
Provable Bayesian Inference via Particle Mirror Descent (2015, background): “8%, to SVI which relies on carefully designed lower bound of the log-likelihood (Jaakkola and Jordan, 1997).”
Affine Independent Variational Inference (2012, background): “4) is the ‘local’ variational bounding procedure [14, 11, 22, 18].”
Hybrid Models with Deep and Invertible Features (2019, background): “In the case of logistic regression, the posterior is no longer conjugate but can be closely approximated (Jaakkola & Jordan, 1997).”