Style Generator Inversion for Image Enhancement and Animation

By Aviv Gabbay, Yedid Hoshen. 2019

17 citations recorded by the source corpus.

One of the main motivations for training high quality image generative models is their potential use as tools for image manipulation. Recently, generative adversarial networks (GANs) have been able to generate images of remarkable quality. Unfortunately, adversarially-trained unconditional generator networks have not been successful as image priors. One of the main requirements for a network to act as a generative image prior, is being able to generate every possible image from the target distribution. Adversarial learning often experiences mode-collapse, which manifests in generators that cannot generate some modes of the target distribution. Another requirement often not satisfied is invertibility i.e. having an efficient way of finding a valid input latent code given a required output image. In this work, we show that differently from earlier GANs, the very recently proposed style-generators are quite easy to invert. We use this important observation to propose style generators as general purpose image priors. We show that style generators outperform other GANs as well as Deep Image Prior as priors for image enhancement tasks. The latent space spanned by style-generators satisfies linear identity-pose relations. The latent space linearity, combined with invertibility, allows us to animate still facial images without supervision. Extensive experiments are performed to support the main contributions of this paper.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “The StyleGAN-based architectures have been applied to numerous applications [38], [39], [40].”
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation (2020, methodology): “By utilizing the weights of pretrained generators, several works [1, 2, 9, 11, 22] design different latent optimization methods to do inpainting, style transfer, morphing, colorization, denoising and super resolution.”
StyleGAN2 Distillation for Feed-forward Image Manipulation (2020, methodology): “The authors use non-trivial loss functions to ﬁnd both close and perceptually good image and show that embedding ﬁts better in extended space W +. Gabbay et al. [15] show that StyleGAN generator can be used as general purpose image prior.”
Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains (2020, methodology): “There are however many methods for encoding an image into the latent space of StyleGAN [1] [12] [5] [10] which could also be used.”
Repurposing GANs for One-Shot Semantic Part Segmentation (2021, background): “However, given any test image, one can optimize for a latent code that generates that given test image with any gradient-based optimization or with more sophisticated schemes [27, 1, 17].”