Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model

By Tian Han, Erik Nijkamp, Xiaolin Fang, Mitch Hill, Song-Chun Zhu, Ying Nian Wu. 2018

74 citations recorded by the source corpus.

This paper proposes the divergence triangle as a framework for joint training of a generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, and contrastive divergence in a unified probabilistic formulation. This unification makes the processes of sampling, inference, and energy evaluation readily available without the need for costly Markov chain Monte Carlo methods. Our experiments demonstrate that the divergence triangle is capable of learning (1) an energy-based model with well-formed energy landscape, (2) direct sampling in the form of a generator network, and (3) feed-forward inference that faithfully reconstructs observed as well as synthesized data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Song-Chun Zhu
2. Arash Vahdat
3. Josh Tenenbaum
4. Yilun Du
5. Ying Nian Wu
6. Sergey Levine
7. Igor Mordatch
8. Jan Kautz

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Song-Chun Zhu
3. Ming-Hsuan Yang
4. Dacheng Tao
5. Trevor Darrell
6. Arash Vahdat
7. Jitendra Malik
8. Xiaogang Wang

In Generative Models

1. Arash Vahdat
2. Song-Chun Zhu
3. Ying Nian Wu
4. Karsten Kreis
5. Erik Nijkamp
6. Jan Kautz
7. Ruiqi Gao
8. Yilun Du

Citations in the corpus

• Flow Contrastive Estimation of Energy-Based Models (2019, background): “CD has been generalized to persistent CD [67], and has more recently been generalized to modified CD [13], adversarial CD [29, 5, 22] with modern CNN structure.”
• On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, background): “Learning a ConvNet potential with the help of a generator network as approximative direct sampler is explored in [17, 5, 31, 32, 10, 18].”
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, methodology): “Pang et al. (2020) and Vahdat et al. (2018b;a; 2020) use EBMs for the prior distribution, and (Han et al., 2020; 2019) jointly learn a VAE and an EBM with independent sets of parameters by an adversarial game.”
• Convolutional Bipartite Attractor Networks (2019, background): “Energy-based models (EBMs) are also density-estimation models that learn a mapping from input data to energies and are trained to assign low energy values to the data manifold [7, 8, 22, 43].”
• NCP-VAE: Variational Autoencoders with Noise Contrastive Priors (2020, methodology): “Han et al. (2019; 2020) have used divergence triangulation to sidesteps MCMC sampling.”
• On Learning Non-Convergent Short-Run MCMC Toward Energy-Based Model (2019, background): “CD has been generalized to persistent CD [44], and has more recently been generalized to modified CD [11] and adversarial CD [21, 5, 16].”
• Improved Contrastive Divergence Training of Energy Based Models (2020, methodology): “In contrast, (Xie et al., 2018a), utilizes a generator to warm start generations for negative samples and (Han et al., 2019) minimizes a divergence triangle be-tween three models.”