A new method for parameter estimation in probabilistic models: Minimum probability flow
By Jascha Narain Sohl-Dickstein, Peter Battaglino, Michael Robert DeWeese. 2020
63 citations recorded by the source corpus.
Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function. We propose a new parameter fitting method, Minimum Probability Flow (MPF), which is applicable to any parametric model. We demonstrate parameter estimation using MPF in two cases: a continuous state space model, and an Ising spin glass. In the latter case it outperforms current techniques by at least an order of magnitude in convergence time with lower error in the recovered coupling parameters.
Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Stefano Ermon
- Yoshua Bengio
- Jun Zhu
- Yang Song
- Sergey Levine
- Quoc V. Le
- Bernhard Schölkopf
- Pieter Abbeel
In Probabilistic & Bayesian Methods
- Stefano Ermon
- Yang Song
- Yoshua Bengio
- Jun Zhu
- Diederik P. Kingma
- Jascha Narain Sohl-Dickstein
- Bernhard Schölkopf
- Chongxuan Li
In Signal Processing
- Bernhard Schölkopf
- Stefano Ermon
- Yoshua Bengio
- Yang Song
- Aapo Hyvärinen
- Jascha Narain Sohl-Dickstein
- Jun Zhu
- Chongxuan Li
Citations in the corpus
- Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, methodology): “The contribution to training from the partition function can be estimated via MCMC [35, 38, 39], via training another generator network [40, 41], or via surrogate objectives to maximum likelihood [42–44].”
- Deep Energy Estimator Networks (2018, methodology): “More recently, minimum probability flow, with deep connections to score matching, was used to learn Ising spin glasses (Sohl-Dickstein et al., 2011).”
- Efficient and optimal binary Hopfield associative memory storage using minimum probability flow (2012, methodology): “The minimum probability flow (MPF) objective function [6] given the collection D is”
- Efficient Learning of Generative Models via Finite-Difference Score Matching (2020, methodology): “As to the more implicit connections to FD, the minimum probability flow (MPF) [56] is a method for parameter estimation in probabilistic models.”