Non-conjugate Variational Message Passing for Multinomial and Binary Regression

By David A. Knowles, Tom Minka. 2011

127 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, methodology): “It has been extended to non-conjugate models (Knowles and Minka 2011) and generalized via factor graphs (Minka 2005).”
Composing graphical models with neural networks for structured representations and fast inference (2016, background): “[25], rather than linearizing intractable terms around the current iterate, in this work we handle intractable terms via recognition networks and amoritized inference (and the remaining tractable objective terms are multi-concave in general, analogous to SVI [10]).”
Task-Agnostic Continual Learning Using Online Variational Bayes With Fixed-Point Updates (2020, methodology): “Fixed-point equations for the variational Bayes optimization problems have been used in the cases of linear models (Knowles & Minka, 2011; Cseke, Opper, & Sanguinetti, 2013) and gaussian processes (Opper & Archambeau, 2009). t However, as we will show here, using nondiagonal covariance can…”
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression (2012, methodology): “This is closely related to the non-conjugate variational message passing (NCVMP) algorithm of Knowles and Minka (2011) implemented in Infer.”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, methodology): “…et al., 2012, Hoffman et al., 2013, Salimans and Knowles, 2013, Sheth and Khardon, 2016a, Sheth et al., 2015, Sheth and Khardon, 2016b]; local VI [Knowles and Minka, 2011, Wand, 2014, Khan and Lin, 2018]; power EP and related algorithms [Minka, 2001, 2004, Li et al., 2015, Hasenclever et al.,…”
Variational Bayesian Inference with Stochastic Search (2012, background): “5 , 1] and w ≥ 0 satisﬁes this requirement.”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, methodology): “The lower bound is not guaranteed to increase at every step in NCVMP and sometimes convergence issues may be encountered which may require damping to ﬁx (see Knowles and Minka, 2011).”
Variational Inference for Heteroscedastic Semiparametric Regression (2015, methodology): “The accommodation of heteroscedasticity is aided by the variational inference technique known as non-conjugate variational message passing (e.g. Knowles & Minka 2011).”
Black Box Variational Inference (2013, methodology): “This hinders us from rapidly exploring modeling assumptions when solving applied problems, and it makes variational methods on complicated distributions impractical for many practitioners.”
Variational inference in nonconjugate models (2012, unknown): “Knowles and Minka (2011) presented a message passing algorithm for non-conjugate models, which has been implemented in Infer.NET (Minka et al., 2010); their technique applies to a subset of models described in this paper.”
Affine Independent Variational Inference (2012, background): “For p(w) in this model class and Gaussian q(w) = N ( w|m,CC ) , B is tighter than ‘local’ bounds [14, 11, 22, 18, 16].”
Variational learning for latent Gaussian model of discrete data (2012, methodology): “Updates of a can be done using an iterated procedure, and hence computation overhead is not much higher than the log bound; see [Knowles and Minka, 2011] for details.”
Continual Learning with Bayesian Neural Networks for Non-Stationary Data (2020, background): “The optimality conditions ∂ µ L ( µ, Σ; D ) = 0 and ∂ Σ L ( µ, Σ; D ) = 0 can be rewritten as follows (Knowles & Minka, 2011; Opper & Archambeau, 2008; Cseke et al., 2013) (cf. App.”
Advances in Variational Inference (2017, unknown): “In contrast, approaches based on additional lower bounds with closed-form updates [82], [88], [207] can”
Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations (2012, methodology): “Knowles and Minka (2011) suggested using damping to ﬁx convergence problems.”
Semiparametric Mean Field Variational Bayes: General Principles and Numerical Issues (2016, methodology): “In Knowles and Minka (2011) the focus is incorporation of pre-specified exponential family distributions whilst preserving the message passing aspect of a modular approach to mean field variational Bayes, known as variational message passing.”