Non-Adversarial Image Synthesis With Generative Latent Nearest Neighbors

By Yedid Hoshen, Jitendra Malik. 2018

59 citations recorded by the source corpus.

Unconditional image generation has recently been dominated by generative adversarial networks (GANs). GAN methods train a generator which regresses images from random noise vectors, as well as a discriminator that attempts to differentiate between the generated images and a training set of real images. GANs have shown amazing results at generating realistic looking images. Despite their success, GANs suffer from critical drawbacks including: unstable training and mode-dropping. The weaknesses in GANs have motivated research into alternatives including: variational auto-encoders (VAEs), latent embedding learning methods (e.g. GLO) and nearest-neighbor based implicit maximum likelihood estimation (IMLE). Unfortunately at the moment, GANs still significantly outperform the alternative methods for image generation. In this work, we present a novel method - Generative Latent Nearest Neighbors (GLANN) - for training generative models without adversarial training. GLANN combines the strengths of IMLE and GLO in a way that overcomes the main drawbacks of each method. Consequently, GLANN generates images that are far better than GLO and IMLE. Our method does not suffer from mode collapse which plagues GAN training and is much more stable. Qualitative results show that GLANN outperforms a baseline consisting of 800 GANs and VAEs on commonly used datasets. Our models are also shown to be effective for training truly non-adversarial unsupervised image translation.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

LaDDer: Latent Data Distribution Modelling with a Generative Prior (2020, unknown): “Thanks to all the preferable properties, VAEs have been widely applied in many computer vision applications, including image synthesis [11, 17, 41], human motion modelling [14, 36] and 3D reconstruction [1, 4, 6, 10, 29].”
Demystifying Inter-Class Disentanglement (2019, methodology): “In line with other work Hoshen & Malik (2019), we found that perceptual losses are very helpful for latent optimization.”
Inclusive GAN: Improving Data and Minority Coverage in Generative Models (2020, background): “Several follow-up works, e.g., GLANN [20] and conditional IMLE [31], also validate the improved effectiveness of IMLE framework in a reconstructive generation context.”
Learning Implicit Generative Models by Matching Perceptual Features (2019, methodology): “[16] obtains better results than GLO by combining it with IMLE [23].”
Generative Latent Flow: A Framework for Non-adversarial Image Generation (2019, methodology): “Note that while achieving comparable or better performances, our method needs significantly fewer training epochs than GLANN and Two-stage VAE (See 5.3.1 and Appendix B).”
Latent Optimization for Non-adversarial Representation Disentanglement (2019, background): “image to image mapping [11], word translation [10]).”
Style Generator Inversion for Image Enhancement and Animation (2019, background): “We therefore believe that other properties such as reanimation could be performed in the future with non-adversarial architectures such as GLANN [12], which are easier and faster to train.”