Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

By Qiang Liu, Dilin Wang. 2016

1,289 citations recorded by the source corpus.

We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Reinforcement Learning with Deep Energy-Based Policies (2017, methodology): “The sampling network trained as part of SVGD can also be viewed as tking the role of an actor in an actor-critic algorithm.”
Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “variational dropout [16, 17], Stein variational gradient descent [18, 9], although other methods exist, e.”
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems (2019, neutral): “2 more tight [32,46–49].”
Bayesian Model-Agnostic Meta-Learning (2018, methodology): “Because a similar instability is also observed in the SVPG paper (Liu & Wang, 2016), we presume that one possible reason is the instability of SVGD such as sensitivity to kernel function parameters.”
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, methodology): “We then introduce three speciﬁc memory evolution methods to solve the Dynamic DRO, including Langevin Dynamics (Welling & Teh, 2011), Stein Variational Gradient Descent (Liu & Wang, 2016), and Hamiltonian ﬂow (Ma et al., 2015; Liu et al., 2019).”
Recent Advances in Autoencoder-Based Representation Learning (2018, background): “…Further, by rearranging terms the resulting objective, they arrive at For tractability reasons, Zhao et al. [5] propose to replace the last term in (10) by other divergences such as Jensen-Shannon divergence (implemented as a GAN [34]), Stein variational gradient [48], or MMD [35] (see Section 2).”
Emergent Complexity via Multi-Agent Competition (2017, methodology): “We found random policy initialization to provide enough diversity between agent policies, however techniques that explicitly encourage diversity [Liu and Wang, 2016] can potentially be incorporated in the future.”
Multiagent Soft Q-Learning (2018, methodology): “Soft Q-Learning solves this issue by using Stein Variational Gradient Descent (SVGD) (Liu and Wang 2016) to approximate the optimal policy through minimizing the KL-divergence: where policy π θ ( ·| s ) is our approximate policy.”
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning (2016, methodology): “Liu & Wang (2016) showed that with Algorithm 1 Amortized SVGD for Problem 1 Set batch size m , step-size scheme { (cid:15) t } and kernel k ( x, x (cid:48) ) .”
Discrete Sequential Prediction of Continuous Actions for Deep RL (2017, methodology): “Existing methods use stochastic neural networks [7] or construct energy models [14] sampled with Stein variational gradient descent [22, 47].”
InfoVAE: Information Maximizing Variational Autoencoders (2017, positive): “The Stein variational gradient [19] is a simple and effective framework for matching a distribution q to p by descending the variational gradient of DKL(q(z)||p(z)).”
Facilitating Bayesian Continual Learning by Natural Gradients and Stein Gradients (2019, methodology): “We applied a RBF kernel in the same manner as described in [9] to the Stein gradients and tested the Stein coresets in both permuted and split tasks, comparing with random and K-center coresets.”
Automated Scalable Bayesian Inference via Hilbert Coresets (2017, methodology): “, 2018) and variational (Dieng et al., 2017; Li and Turner, 2016; Liu and Wang, 2016) approaches.”
Stein Variational Policy Gradient (2017, methodology): “Liu and Wang (2016) showed that this functional optimization yields a closed form solution, where k ( x, x (cid:48) ) is the positive deﬁnite kernel associated with the RKHS H 0 .”
Continuous-Time Flows for Efficient Inference and Density Estimation (2017, background): “Both (Wang & Liu, 2017) and (Pu et al., 2017) extend the idea of Stein variational gradient descent (Liu & Wang, 2016) with amortized inference for a GAN-based and a VAE-based model, respectively, which resemble our proposed MacVAE and MacGAN in concept.”
Distributionally Adversarial Attack (2018, methodology): “The bandwidth is set as h = med(2)/ logM , same as the kernel used in (Liu and Wang, 2016) and (Chen et al.”
Action-depedent Control Variates for Policy Optimization via Stein's Identity (2017, background): “An incomplete list includes Gorham & Mackey (2015), Oates et al. (2017), Oates et al. (2016), Chwialkowski et al. (2016), Liu et al. (2016), Sedghi et al. (2016), Liu & Wang (2016), Feng et al. (2017), Liu & Lee (2017).”
Active Domain Randomization (2019, methodology): “SVPG uses Stein Variational Gradient Descent (Liu & Wang, 2016) to iteratively update an ensemble of N policies or particles µ φ = { µ φ i } Ni =1 with an update rule: with step size (cid:15) and positive deﬁnite kernel k .”
A Bayesian Approach to Generative Adversarial Imitation Learning (2018, methodology): “SVGD [Liu and Wang, 2016] is a recently proposed Bayesian inference algorithm based on the particle updates, which we briefly review as follows: suppose that random variable x follows the distribution q(0), and target distribution p is known up to the normalization constant.”