PixelGAN Autoencoders

By Alireza Makhzani, Brendan J. Frey. 2017

102 citations recorded by the source corpus.

In this paper, we describe the "PixelGAN autoencoder", a generative autoencoder in which the generative path is a convolutional autoregressive neural network on pixels (PixelCNN) that is conditioned on a latent code, and the recognition path uses a generative adversarial network (GAN) to impose a prior distribution on the latent code. We show that different priors result in different decompositions of information between the latent code and the autoregressive decoder. For example, by imposing a Gaussian distribution as the prior, we can achieve a global vs. local decomposition, or by imposing a categorical distribution as the prior, we can disentangle the style and content information of images in an unsupervised fashion. We further show how the PixelGAN autoencoder with a categorical prior can be directly used in semi-supervised settings and achieve competitive semi-supervised classification results on the MNIST, SVHN and NORB datasets.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

Progressive Growing of GANs for Improved Quality, Stability, and Variation (2017, methodology): “Hybrid methods combine various strengths of the three, but so far lag behind GANs in image quality (Makhzani & Frey, 2017; Ulyanov et al., 2017; Dumoulin et al., 2016).”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, methodology): “Other generative models based on similar principles are Variational Recurrent AEs [85], PixelGAN AEs [86] and Adversarial Symmetric Variational AEs [87].”
Disentangling by Factorising (2018, methodology): “In PixelGAN Autoencoders (Makhzani & Frey, 2017), the same objective is used to study the decomposition of information between the latent code and the decoder.”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “VAE D KL ( q φ ( z | x ) (cid:107) p ( z )) VIB [9] VAE D KL ( q φ ( z | x ) (cid:107) p ( z )) O PixelGAN-AE[18] VAE − I q φ ( x ; z ) O InfoVAE [5] VAE D KL ( q φ ( z | x ) (cid:107) p ( z )) D KL ( q φ ( z ) (cid:107) p ( z )) Info. dropout [10] VAE D KL ( q φ ( z | x ) (cid:107) p ( z )) TC ( q…”
Generative Probabilistic Novelty Detection with Adversarial Autoencoders (2018, background): “Similarly to AAEs, PixelGAN autoencoders [45] introduce the adversarial component to impose a prior distribution on the latent code, but the architecture is signiﬁcantly different, since it is conditioned on the latent code.”
InfoGCN: Representation Learning for Human Skeleton-based Action Recognition (2022, background): “(5) can be further decomposed following [16, 34].”
Learning joint latent representations based on information maximization (2021, background): “To understand why increasing bwould help disengagements, some studies [18,19] have decomposed the KL divergence term into two components:”
Hierarchical Adversarially Learned Inference (2018, methodology): “17 0 LMLE-kNN (Huang et al., 2016a) MNIST (# errors) VAE (M1+M2) (Kingma et al., 2014) 233 ± 14 VAT (Miyato et al., 2017) 136 CatGAN (Springenberg, 2015) 191 ± 10 Adversarial Autoencoder (Makhzani et al., 2015) 190 ± 10 PixelGAN (Makhzani & Frey, 2017) 108 ± 15 ADGM (Maaløe et al., 2016) 96 ± 2 Feat”
Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences (2020, background): “…(3) Learn an embedding φ ( s ) that predicts the temporal distance between two randomly chosen states from the same demonstration (Aytar et al., 2018), and (4) Train a variational pixel-to-pixel autoencoder in which φ ( s ) is the learned latent encoding (Makhzani & Frey, 2017; Doersch, 2016).”
Joint-VAE: Learning Disentangled Joint Continuous and Discrete Representations (2018, background): “The KL term then becomes (Makhzani & Frey (2017); Kim & Mnih (2018)) i.e., when taken in expectation over the data, the KL divergence term is an upper bound on the mutual information between the latents and the data (see appendix for proof and details).”