Disentangled Inference for GANs With Latently Invertible Autoencoder

By Jiapeng Zhu, Deli Zhao, Bo Zhang, Bolei Zhou. 2019

39 citations recorded by the source corpus.

Generative Adversarial Networks (GANs) can synthesize more and more realistic images. However, one fundamental issue hinders their practical applications: the incapability of encoding real samples in the latent space. Many semantic image editing applications rely on inverting the given image into the latent space and then manipulating inverted code. One possible solution is to learn an encoder for GAN via Variational Auto-Encoder. However, the entanglement of the latent space poses a major challenge for learning the encoder. To tackle the challenge and enable inference in GANs, we propose a novel method named Latently Invertible Autoencoder (LIA). In LIA, an invertible network and its inverse mapping are symmetrically embedded in the latent space of an autoencoder. The decoder of LIA is first trained as a standard GAN with the invertible network, and then the encoder is learned from a disentangled autoencoder by detaching the invertible network from LIA. It thus avoids the entanglement problem caused by the latent space. Extensive experiments on the FFHQ face dataset and three LSUN datasets validate the effectiveness of LIA for the image inversion and its applications. Code and models are available at https://github.com/genforce/lia.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Peihao Zhu
2. Peter Wonka
3. Rameen Abdal
4. Niloy J. Mitra
5. Daniel Cohen-Or
6. Ming-Hsuan Yang
7. Eli Shechtman
8. Luc Van Gool

In Generative Models

1. Peihao Zhu
2. Peter Wonka
3. Rameen Abdal
4. Niloy J. Mitra
5. Daniel Cohen-Or
6. Yotam Nitzan
7. Or Patashnik
8. Yuval Alaluf

In Representation & Self-Supervised Learning

1. Peihao Zhu
2. Niloy J. Mitra
3. Peter Wonka
4. Rameen Abdal

Citations in the corpus

• StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “Finally, other methods (Zhu et al. 2019b) use VAEs to create inverse mappings.”