Deep Unsupervised Learning using Nonequilibrium Thermodynamics

By Jascha Narain Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli. 2015

10,395 citations recorded by the source corpus.

A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Sergey Levine
  3. Mohit Bansal
  4. Jianfeng Gao
  5. Quoc V. Le
  6. Percy Liang
  7. Ruslan Salakhutdinov
  8. Trevor Darrell

In Statistical and Nonlinear Physics

  1. Arash Vahdat
  2. Stefano Ermon
  3. Yang Song
  4. Jonathan Ho
  5. Karsten Kreis
  6. Jiaming Song
  7. Diederik P. Kingma
  8. Prafulla Dhariwal

In Generative Models

  1. Stefano Ermon
  2. Arash Vahdat
  3. Jun-Yan Zhu
  4. Jiaming Song
  5. Richard Zhang
  6. Karsten Kreis
  7. Yoshua Bengio
  8. Eli Shechtman

In Probabilistic & Bayesian Methods

  1. Stefano Ermon
  2. Yoshua Bengio
  3. Yang Song
  4. Jiaming Song
  5. Diederik P. Kingma
  6. Arash Vahdat
  7. Aaron C. Courville
  8. Max Welling

Citations in the corpus