Divergence measures and message passing

By Tom Minka. 2005

663 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (2016, methodology): “To demonstrate the properties of the different f -divergences and to validate the variational divergence estimation framework we perform an experiment similar to the one of [24].”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “For example, inclusive KL encourages the coverage of the support set (referred as mass-covering), and when used in lo-cal divergence minimisation (Minka, 2005), it can ﬁt an approximation to a mode of p ( ω ) with better estimates of uncertainty.”
Learning in Implicit Generative Models (2016, background): “Objective Function Hinge loss where the ﬁrst equation is the KL for un-normalised densities (Minka, 2005).”
Stabilizing Training of Generative Adversarial Networks through Regularization (2017, background): “Here, the principle of parameter estimation via divergence minimization is provably sound in that it achieves a well-defined limit [1, 21].”
Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks (2016, neutral): “We aim to adjust q by minimizing the α-divergence between p and q [13]: Dα[p||q] = 1 α(α− 1) ( 1− ∫ p(W)q(W) ) dW , (5)”
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression (2012, background): “It is for example well known (e.g. Minka, 2005; Nickisch and Rasmussen, 2008; Turner et al., 2008) that the tails of variational posterior approximations tend to be thinner than those of the actual posterior unless the approximation is extremely close, which means that using EIS with the…”
Black-box $\alpha$-divergence Minimization (2015, background): “Both VB and EP are speciﬁc cases of local α -divergence minimization, where the parameter α ∈ ( −∞ , + ∞ ) \ { 0 } speciﬁes the properties of the divergence to be minimized (Minka, 2005).”
Expectation propagation for neural networks with sparsity-promoting priors (2013, background): “…sites withθ fixed to a sufficiently small value, such asσ2 = exp(θ) = 0.32, and the input weight priors set to ˜µw, j = 0 andσ̃2w, j = 0.5, where we have assumed that the target variablesy and the columns ofX containing the input variables are normalized to zero mean and unit variance.”
Importance Weighting and Variational Inference (2018, background): “Our result makes it clear exactly what KL-divergence is being minimized by maximizing the IW-ELBO and in what sense doing this makes q “close to” p .”
Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities (2010, methodology): “It will be also shown how the Alpha-divergences are derived from the Csiszár–Morimoto f -divergence and the Beta-divergence form the Bregman divergence by using the power functions.”
Deep Label Distribution Learning With Label Ambiguity (2016, neutral): “…are (+77◦,-4◦) (-16◦,-1◦) (-1◦,-30◦) (+30◦,+8◦) (+4◦,-4◦) (-36◦,+13◦) (-87◦,-3◦) (-61◦,-58◦) (+63◦,+12◦) (+80◦,-27◦) −90 −72 −54 −36 −18 0 18 36 54 72 90 90 72 54 36 18 0 −18 −36 −54 −72 −90 0 0.01 0.02 0.03 0.04 YawPitch P ro b a b ili ty (+75◦,-3◦) −90 −72 −54 −36 −18 0 18 36 54 72 90 90 72 54…”
Uncertainty in Deep Learning (2016, methodology): “Closely related to PBP are the recently suggested approximate inference techniques based on α-divergence minimisation [Hernández-Lobato et al., 2016; Li and Turner, 2016; Minka, 2005].”
f-Divergence constrained policy improvement (2017, background): “…as f -divergence minimization (Altun and Smola, 2006) and various message passing algorithms can be understood as minimizing different α-divergences (Minka, 2005),—pave the way to new algorithms that are better suited for particular problems—e.g., the generative adversarial networks (GANs)…”
Evaluating probabilities under high-dimensional latent variable models (2008, background): “Specifically, the variance of the estimator is an α-divergence between the distributions [3].”