Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE

By Qiang Liu, Dilin Wang. 2017

23 citations recorded by the source corpus.

We propose a number of new algorithms for learning deep energy models and demonstrate their properties. We show that our SteinCD performs well in term of test likelihood, while SteinGAN performs well in terms of generating realistic looking images. Our results suggest promising directions for learning better models by combining GAN-style methods with traditional energy-based learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Arthur Gretton
2. Le Song
3. Bernhard Schölkopf
4. Bo Dai
5. Kenji Fukumizu
6. Yoshua Bengio
7. Sergey Levine
8. Hanjun Dai

In Computer Vision and Pattern Recognition

1. Le Song
2. Arthur Gretton
3. Bernhard Schölkopf
4. Bo Dai
5. Luc Van Gool
6. Dacheng Tao
7. Hanjun Dai
8. Stefano Ermon

In Generative Models

1. Le Song
2. Hanjun Dai
3. Bernhard Schölkopf
4. Bo Dai
5. Arthur Gretton
6. Dale Schuurmans
7. Aapo Hyvärinen
8. Arash Mehrjou

In Probabilistic & Bayesian Methods

1. Arthur Gretton
2. Bernhard Schölkopf
3. Kenji Fukumizu
4. Bharath K. Sriperumbudur
5. Le Song
6. Krikamol Muandet
7. Bo Dai
8. Hanjun Dai

Citations in the corpus

• Deep Energy Estimator Networks (2018, methodology): “In preparing this submission, we came across the work by Liu & Wang (2017) and the Stein contrastive divergence introduced there for learning energy models.”
• Exponential Family Estimation via Adversarial Dynamics Embedding (2019, methodology): “Exploiting deep models for energy-based model estimation has been investigated in Kim and Bengio (2016); Dai et al. (2017); Liu and Wang (2017); Dai et al. (2019).”