StyleGAN2 Distillation for Feed-forward Image Manipulation

By Yuri Viazovetskyi, Vladimir Ivashkin, Evgenii Kashin. 2020

158 citations recorded by the source corpus.

StyleGAN2 is a state-of-the-art network in generating realistic images. Besides, it was explicitly trained to have disentangled directions in latent space, which allows efficient image manipulation by varying latent factors. Editing existing images requires embedding a given image into the latent space of StyleGAN2. Latent code optimization via backpropagation is commonly used for qualitative embedding of real world images, although it is prohibitively slow for many applications. We propose a way to distill a particular image manipulation of StyleGAN2 into image-to-image network trained in paired way. The resulting pipeline is an alternative to existing GANs, trained on unpaired data. We provide results of human faces' transformation: gender swap, aging/rejuvenation, style transfer and image morphing. We show that the quality of generation using our method is comparable to StyleGAN2 backpropagation and current state-of-the-art methods in these particular tasks.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP (2022, neutral): “The image search domain used by these methods could be the latent codes of a pre-trained GAN model (e.g. BigGAN [1], StyleGAN [14, 24], SWAGAN [4]), the codebook of a VQGAN model [7], Diffusion Denoising Models [3], structured representations such as a set of strokes (ClipDraw [13]) or triangles [6]”
GAN Inversion: A Survey (2021, methodology): “Other GAN inversion methods [54] based on pretrained StyleGAN [13] or StyleGAN2 [14] models can also manipulate multiple attributes due to the stronger separability of W space than of Z space.”
InstructPix2Pix: Learning to Follow Image Editing Instructions (2022, background): “As generative models continue to improve, there is growing interest in their use as a source of cheap and plentiful training data for downstream tasks [32, 46, 49, 57, 64, 65].”
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data (2023, methodology): “GANs [30] have been used widely for dataset generation on tasks such as visual alignment [63], face manipulation [86], and adversarial training for image synthesis [77].”
Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation (2021, unknown): “The global directions are commonly used (Härkönen et al., 2020; Jahanian et al., 2019; Shen et al., 2019; Viazovetskyi et al., 2020).”
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs (2020, background): “that we cannot manipulate the attribute in a continuous manner [69].”
EditGAN: High-Precision Semantic Image Editing (2021, methodology): “2, we also compare with StyleGAN2 Distillation [82], which achieves strong performance.”
Only a matter of style: age transformation using a style-based regression model (2021, background): “Motivated by the early successes of these works, many methods [25, 27, 43, 54, 65] have approached the task of face aging as an image-to-image translation between multiple age groups.”
Ensembling with Deep Generative Views (2021, background): “Several works investigate using generative samples as data augmentation in an image translation setting, in which an autoencoder-style network is trained to produce variations of an input image, on a variety of domains such as face attribute editing [57], gaze estimation [54], emotion classification”
Pixel-aligned Volumetric Avatars (2021, background): “Image-based Methods Recently, there has been a great deal of progress in high-quality controllable face synthesis [7, 30, 1].”
Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP (2021, methodology): “Inspired by recent work in knowledge distillation for image manipulation with StyleGAN [17] we generate a synthetic dataset with images and corresponding segmentation masks and train a simple segmentation model on that dataset.”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “The model achieves remarkable image synthesis quality and has found important applications for example in image editing [88].”