Estimation of Non-Normalized Statistical Models by Score Matching

By Aapo Hyvärinen. 2005

2,147 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Generalized Denoising Auto-Encoders as Generative Models (2013, neutral): “L G ] 7 J un 2 01 3 solution as score matching (Hyvärinen, 2005).”
Neural Autoregressive Distribution Estimation (2016, background): “Other proposals include pseudo-likelihood (Besag, 1975; Marlin et al., 2010), score matching (Hyv¨arinen, 2005; Hyv¨arinen, 2007a,b), noise contrastive estimation (Gutmann and Hyv¨arinen, 2010) and probability ﬂow minimization (Sohl-Dickstein et al., 2011).”
Improved Techniques for Training Score-Based Generative Models (2020, methodology): “First, the score function is learned via denoising score matching [3, 4, 5].”
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, background): “, 2019), or via surrogate objectives to maximum likelihood (Hyvärinen, 2005; Gutmann & Hyvrinen, 2010; Sohl-Dickstein et al., 2011).”
Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, methodology): “Score matching (Hyvärinen, 2005) attempts to match the derivative of the model’s density with the derivative of the data density.”
Flow Contrastive Estimation of Energy-Based Models (2019, background): “Another line of work is to estimate the parameters of EBM by score matching [26, 74, 63, 64].”
Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, methodology): “R � To ﬁt the model, many approximate methods have been developed ( e.g. , pseudo log-likelihood [6], contrastive divergence [29, 65], score matching estimator [30], minimum probability ﬂow [64], noise contrastive estimation [27]) to bypass the calculation of the partition function.”
Generative Modeling by Estimating Gradients of the Data Distribution (2019, methodology): “, Theorem 2 in [24]), and will be inconsistent when the data reside on a low-dimensional manifold.”
A Theory of Generative ConvNet (2016, methodology): “Proposition 3 is related to score matching estimator (Hyvärinen, 2005), whose connection with contrastive divergence based on one-step Langevin was studied by (Hyvärinen, 2007).”
Score-based Generative Modeling in Latent Space (2021, methodology): “Our work builds on score-matching [29, 30, 31, 32, 33, 34, 35, 36, 37], specifically denoising score matching [22], which makes our work related to recent generative models using denoising score matching- and denoising diffusion-based objectives [3, 38, 1, 2, 6].”
Image Super-Resolution via Iterative Refinement (2021, methodology): “Score matching [35] models the gradient of the data log-”
Noise Estimation for Generative Diffusion Models (2021, methodology): “In [31], a new generative model based on the score matching method [10] and Langevin dynamics was introduced.”
Score-Based Generative Modeling through Stochastic Differential Equations (2020, positive): “The score of a distribution can be estimated by training a score-based model on samples with score matching (Hyv¨arinen, 2005; Song et al., 2019a .”
Denoising Diffusion Implicit Models (2020, methodology): “(2020), the objective with γ = 1 is optimized instead to maximize generation performance of the trained model; this is also the same objective used in noise conditional score networks (Song & Ermon, 2019) based on score matching (Hyvärinen, 2005; Vincent, 2011).”
WaveGrad: Estimating Gradients for Waveform Generation (2020, methodology): “The Stein score function (Hyvärinen, 2005) is the gradient of the data log-density log p(y) with respect to the datapoint y:”
Improved Denoising Diffusion Probabilistic Models (2021, background): “More recently, Ho et al. (2020) showed an equivalence between denoising diffusion probabilistic models (DDPM) and score based generative models (Song & Ermon, 2019; 2020), which learn a gradient of the log-density of the data distribution using denoising score matching (Hyv¨arinen, 2005).”