Joint Training of Variational Auto-Encoder and Latent Energy-Based Model

By Tian Han, Erik Nijkamp, Linqi Zhou, Bo Pang, Song-Chun Zhu, Ying Nian Wu. 2020

63 citations recorded by the source corpus.

This paper proposes a joint training method to learn both the variational auto-encoder (VAE) and the latent energy-based model (EBM). The joint training of VAE and latent EBM are based on an objective function that consists of three Kullback-Leibler divergences between three joint distributions on the latent vector and the image, and the objective function is of an elegant symmetric and anti-symmetric form of divergence triangle that seamlessly integrates variational and adversarial learning. In this joint training scheme, the latent EBM serves as a critic of the generator model, while the generator model and the inference model in VAE serve as the approximate synthesis sampler and inference sampler of the latent EBM. Our experiments show that the joint training greatly improves the synthesis quality of the VAE. It also enables learning of an energy function that is capable of detecting out of sample examples for anomaly detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Arash Vahdat
2. Karsten Kreis
3. Jan Kautz
4. Alexander Gerhard Schwing
5. Jyoti Aneja
6. Zhisheng Xiao
7. Raquel Urtasun
8. Rewon Child

In Computer Vision and Pattern Recognition

1. Arash Vahdat
2. Karsten Kreis
3. Jan Kautz
4. Luc Van Gool
5. Alexander Gerhard Schwing
6. Ming-Hsuan Yang
7. Dacheng Tao
8. Zhisheng Xiao

In Generative Models

1. Karsten Kreis
2. Arash Vahdat
3. Jan Kautz
4. Zhisheng Xiao
5. Alexander Gerhard Schwing
6. Jyoti Aneja
7. Robin Rombach
8. Jiaming Song

Citations in the corpus

• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, methodology): “Pang et al. (2020) and Vahdat et al. (2018b;a; 2020) use EBMs for the prior distribution, and (Han et al., 2020; 2019) jointly learn a VAE and an EBM with independent sets of parameters by an adversarial game.”
• Learning Energy-Based Models by Diffusion Recovery Likelihood (2020, neutral): “02 EBM-Triangle (Han et al., 2020) 24.”
• Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models (2020, unknown): “53 , which is better than baselines such as VAE-EBLVM (Han et al., 2020) ( 7 .”
• NCP-VAE: Variational Autoencoders with Noise Contrastive Priors (2020, methodology): “Han et al. (2019; 2020) have used divergence triangulation to sidesteps MCMC sampling.”