Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling

By Will Grathwohl, Kuan-Chieh Jackson Wang, Jörn-Henrik Jacobsen, David Kristjanson Duvenaud, Richard S. Zemel. 2020

100 citations recorded by the source corpus.

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “Joint EBMs (Grathwohl et al., 2020a; Liu & Abbeel, 2020) model the joint distribution of data and labels.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “Since score estimates have high variance, the Stein discrepancy has been proposed as an alternative objective, requiring no sampling and more closely correlating with likelihood [63].”
Diffusion policy: Visuomotor policy learning via action diffusion (2023, unknown): “Implicit policies (Florence et al., 2021; Jarrett et al., 2020) de ﬁ ne distributions over actions by using Energy-Based Models (EBMs) (Dai et al., 2019; Du et al., 2020; Du and Mordatch, 2019; Grathwohl et al., 2020; LeCun et al., 2006).”
Is Conditional Generative Modeling all you need for Decision-Making? (2022, methodology): “The denoising procedure can be alternatively interpreted as parameterizing the gradients of the data distribution (Song et al., 2021) optimizing the score matching objective (Hyv¨arinen, 2005) and thus as a Energy-Based Model (Du & Mordatch, 2019; Nijkamp et al., 2019; Grathwohl et al., 2020).”
Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models (2020, unknown): “1 (a), we numerically validate Theorem 1,2 in Gaussian restricted Boltzmann machines (GRBMs) (Hinton & Salakhutdinov, 2006).”
DreamFusion: Text-to-3D using 2D Diffusion (2022, background): “GAN-like amortized samplers can be learned by minimizing the Stein discrepancy (Hu et al., 2018; Grathwohl et al., 2020), where the optimal critic resembles the difference of scores in our loss (Eqn.”
Planning with Diffusion for Flexible Behavior Synthesis (2022, methodology): “The denoising procedure can be seen as parameterizing the gradients of the data distribution (Song & Ermon, 2019), connecting diffusion models to score matching (Hyv¨arinen, 2005) and EBMs (LeCun et al., 2006; Du & Mordatch, 2019; Nijkamp et al., 2019; Grathwohl et al., 2020).”
Compositional Visual Generation with Composable Diffusion Models (2022, methodology): “The denoising procedure can be seen as parameterizing the gradients of the data distribution [40], connecting diffusion models to EBMs [7, 11, 12, 22, 30].”