It Takes (Only) Two: Adversarial Generator-Encoder Networks

By Dmitry Ulyanov, Andrea Vedaldi, Victor S. Lempitsky. 2017

141 citations recorded by the source corpus.

We present a new autoencoder-type architecture that is trainable in an unsupervised mode, sustains both generation and inference, and has the quality of conditional and unconditional samples boosted by adversarial learning. Unlike previous hybrids of autoencoders and adversarial networks, the adversarial game in our approach is set up directly between the encoder and the generator, and no external mappings are trained in the process of learning.The game objective compares the divergences of each of the real and the generated data distributions with the prior distribution in the latent space. We show that direct generator-vs-encoder game leads to a tight coupling of the two components, resulting in samples and reconstructions of a comparable quality to some recently-proposed more complex architectures.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Large Scale Adversarial Representation Learning (2019, background): “Other hybrid approaches like AGE [31] and α-GAN [27] add an encoder to stabilize GAN training.”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, methodology): “Inaddition to the approach of combining the autoen-coder and adversarial networks mentioned above, Ulyanov et al. [19] proposed an approach named Adversarial Generator-Encoder Network (AGE), in which the adversarial network acts between the generator and the encoder, and the network does not…”
Adversarial Text Generation Without Reinforcement Learning (2018, background): “An encoder which maps generated sentence vectors back into Gaussian vectors could improve the smoothness and potential applications of the space (Ulyanov et al., 2017).”
Wasserstein Auto-Encoders (2017, background): “es and use the learned manifold, in which cases these models are not applicable. There have been many other approaches trying to blend the adversarial training of GANs with auto-encoder architectures [17, 18, 19, 20]. The approach proposed by [19] is perhaps the most relevant to our work. The author”
Adversarial Latent Autoencoders (2020, background): “For the latent space instead, the common practice is to set a desired target latent distribution, and then the encoder is trained to match it either by minimizing a divergence type of similarity [28, 41, 31, 47, 48], or by setting up an adversarial game [35, 6, 9, 49, 20].”
An Overview of Deep Semi-Supervised Learning (2020, background): “This ( C + 1) -class discriminator objective lead to strong empirical results, and was later widely used to evaluate the eﬀectiveness of generative models [42, 152].”
Guided Image Generation with Conditional Invertible Neural Networks (2019, methodology): “Conceptually closest to our proposed method is the work of [39], where an encoder network maps color information to a latent space and a generator network learns the inverse transform, both conditioned on the grayscale image.”
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, background): “[78, 79] focus on inverse mapping - projecting data back into the latent space, resulting in a useful feature representation for auxiliary discrimination tasks.”
Towards Photographic Image Manipulation with Balanced Growing of Generative Autoencoders (2019, methodology): “…in the limited C ELEB A-HQ training set, such as a dark-skinned face with atypical orientation, will result in more pronounced failures.) coder (IntroVAE, [16]), and the progressively growing generative autoencoder (P IONEER , [13]) based on the Adversarial Generator–Encoder (AGE, [40]).”
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition (2019, methodology): “The more recent IntroVAE [13] and adversarial encoder generator networks [14] have subsequently come to the realization that this doesn’t necessarily require the auxiliary real-fake discriminator, as the VAE itself already provides strong means for discrimination, namely its probabilistic encoder.”
IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis (2018, methodology): “Compared to AGE [36], the major differences are addressed in three-fold.”
Disentangled Inference for GANs With Latently Invertible Autoencoder (2019, unknown): “Thesemetrics are commonly used to measure the quality of the images in GANs (Ulyanov et al. 2017; Karras et al. 2018a; Donahue et al. 2017; Karras et al. 2018b).”
Controllable Continuous Gaze Redirection (2020, result): “also introduce a knowledge distillation loss to guide the training of �. Knowledge distillation [3] is widely used to compress CNN models and has recently been utilized for latent space interpolation [7, 31]. However, distilling the dark knowledge from the teacher’s output pixels is difficult. Inste”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “Combination of a generator and an encoder: Different from previous hybrids of autoencoders and GANs, Adversarial Generator-Encoder (AGE) Network [42] is set up directly between the generator and the encoder, and no external mappings are trained in the learning process.”