Adversarial Latent Autoencoders

By Stanislav Pidhorskyi, D. Adjeroh, Gianfranco Doretto. 2020

289 citations recorded by the source corpus.

Autoencoder networks are unsupervised approaches aiming at combining generative and representational properties by learning simultaneously an encoder-generator map. Although studied extensively, the issues of whether they have the same generative power of GANs, or learn disentangled representations, have not been fully addressed. We introduce an autoencoder that tackles these issues jointly, which we call Adversarial Latent Autoencoder (ALAE). It is a general architecture that can leverage recent improvements on GAN training procedures. We designed two autoencoders: one based on a MLP encoder, and another based on a StyleGAN generator, which we call StyleALAE. We verify the disentanglement properties of both architectures. We show that StyleALAE can not only generate 1024x1024 face images with comparable quality of StyleGAN, but at the same resolution can also produce face reconstructions and manipulations based on real images. This makes ALAE the first autoencoder able to compare with, and go beyond the capabilities of a generator-only type of architecture.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Score-based Generative Modeling in Latent Space (2021, background): “[87] Stanislav Pidhorskyi, Donald A Adjeroh, and Gianfranco Doretto.”
Taming Transformers for High-Resolution Image Synthesis (2020, neutral): “9 Style ALAE [49] 19.”
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, methodology): “Generally, inversion methods either directly optimize the latent vector to minimize the error for the given image [25, 8, 1, 2], train an encoder to map the given image to the latent space [31, 8, 32, 12, 29], or use a hybrid approach combining both [4, 42].”
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation (2020, background): “The en-coder’s structure follows that of StyleALAE [25].”
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022, methodology): “In the GAN literature, this inversion is done through either an optimization-based technique (Abdal et al., 2019, 2020; Zhu et al., 2020b; Gu et al., 2020) or by using an encoder (Richardson et al., 2020; Zhu et al., 2020a; Pidhorskyi et al., 2020; Tov et al., 2021).”
Perception Prioritized Training of Diffusion Models (2022, unknown): “…VDVAE ICLR’21 [5] VAE 33.5 VQGAN CVPR’21 [11] GAN+AR 9.6 D2C NeurIPS’21 [38] Diff [20] GAN 64.70 MSG-GAN CVPR’20 [16] [22] Flow 68.93 ALAE CVPR’20 [33] GAN 19.21 NVAE NeurIPS’20 [44] VAE 29.76 VAEBM ICLR’21 [51] VAE+EM 20.38 VQGAN CVPR’21 [11] GAN+AR 10.70 LSGM NeurIPS’21 [45] VAE+Diff 7.22 P2…”
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs (2021, methodology): “LAE (Pidhorskyi et al., 2020) 19.2 VQ-GAN (Esser et al., 2021b) 10.2 DC-AE (Parmar et al., 2021) 15.8 Table 5: Generative results on LSUN Church 256 Model FID↓ Denoising Diffusion GAN (ours) 5.25 DDPM (Ho et al., 2020) 7.89 ImageBART (Esser et al., 2021a) 7.32 Gotta Go Fast (Jolicoeur-Martineau et a”
A Free Viewpoint Portrait Generator with Dynamic Styling (2020, background): “Since the disentangled representation is discovered[34, 27, 15] in common VAE, GANs[11, 12] architecture, many studies on latent space[33, 28, 24] that observes the vector arithmetic phenomenon have been engaged in changing the attributes of output images.”
Disentangled Inference for GANs With Latently Invertible Autoencoder (2019, result): “We are aware that a concurrent work, called Adversarial Latent Auto-Encoders (ALAE) (Pidhorskyi et al., 2020) proposed a similar idea to ours.”
Only a matter of style: age transformation using a style-based regression model (2021, methodology): “Specifically, latent space methods follow an “invert then edit” methodology in which real images are first inverted into the latent space of a pre-trained GAN [11, 12, 21, 55, 57, 69, 77].”
Face identity disentanglement via latent space mapping (2020, methodology): “Reconstruction quantitative comparison with ALAE [Pidhorskyi et al. 2020] and pSp [Richardson et al. 2020] was devised for the task of disentanglement rather than reconstruction.”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, methodology): “Generally, inversion methods either directly optimize the latent vector to minimize the error for a given image [1, 2, 7, 14, 21, 40, 70, 74, 75], train an encoder over a large number of samples to learn a mapping from an image to its latent representation [5,22,22,30, 36, 43, 49, 50, 53, 61, 65], o”
High-fidelity GAN Inversion with Padding Space (2022, background): “Active attempts broadly fall into two categories, which are optimization-based [8,13,19,31,34,37] and learning-based [2,35,36, 38, 44, 49, 56–58].”
Pivotal Tuning for Latent-based Editing of Real Images (2021, background): “[29] train both the encoder and the StyleGAN generator, but their reconstruction results suffer from significant distortion, as the StyleGAN tuning step is too extensive.”
LOHO: Latent Optimization of Hairstyles via Orthogonalization (2021, background): “In parallel, recent works have achieved impressive latent space manipulation by learning disentangled feature representations [26], enabling photorealistic global and local image manipulation.”