Max-Margin Deep Generative Models

By Chongxuan Li, Jun Zhu, Tianlin Shi. 2015

34 citations recorded by the source corpus.

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chunyuan Li
2. Zhe Gan
3. Lawrence Carin
4. Jun Zhu
5. Yunchen Pu
6. Ricardo Henao
7. Jianfeng Gao
8. Chongxuan Li

In Computer Vision and Pattern Recognition

1. Chunyuan Li
2. Lawrence Carin
3. Zhe Gan
4. Jun Zhu
5. Yunchen Pu
6. Dacheng Tao
7. Luc Van Gool
8. Jianfeng Gao

In Signal Processing

1. Chunyuan Li
2. Lawrence Carin
3. Yunchen Pu
4. Xin Yuan
5. Andrew Stevens
6. Zhe Gan
7. Ricardo Henao
8. Jun Zhu

Citations in the corpus

• Towards Better Analysis of Deep Convolutional Neural Networks (2016, background): “Recently, during the research triggered by [36], E3 tried to construct a variant of BaseCNN.”
• Variational Autoencoder for Deep Learning of Images, Labels and Captions (2016, methodology): “Our DGDN-CNN model for images is related to but distinct from prior convolutional variational auto-encoder networks [13, 14, 15].”
• A Deep Generative Deconvolutional Image Model (2015, methodology): “Table 2: Average test log-likelihood (in nats) on MNIST Method Test log-likelihood CMMVA (Li et al., 2015) -95.”
• Triple Generative Adversarial Networks (2019, background): “In fact, on one hand, the representations learned by unsupervised deep generative models (DGMs) (including GANs) can be less discriminative than those learned by supervised deep neural networks (DNNs) and the predictions made based on the representations are not sufficiently accurate [21].”
• On the Depth of Deep Neural Networks: A Theoretical View (2015, background): “One related work is (Li et al. 2015), which combines the generative deep learning methods (e.”