Improved StyleGAN Embedding: Where are the Good Latents?

By Peihao Zhu, Rameen Abdal, Yipeng Qin, Peter Wonka. 2020

127 citations recorded by the source corpus.

StyleGAN is able to produce photorealistic images almost indistinguishable from real ones. Embedding images into the StyleGAN latent space is not a trivial task due to the reconstruction quality and editing quality trade-off. In this paper, we first introduce a new normalized space to analyze the diversity and the quality of the reconstructed latent codes. This space can help answer the question of where good latent codes are located in latent space. Second, we propose a framework to analyze the quality of different embedding algorithms. Third, we propose an improved embedding algorithm based on our analysis. We compare our results with the current state-of-the-art methods and achieve a better trade-off between reconstruction quality and editing quality.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Designing an encoder for StyleGAN image manipulation (2021, background): “Previous works [46], and [45] also observed the fact that different regions in the latent space have different editing properties and presented inversion methods that explicitly addressed this.”
Paint by Word (2021, background): “Because several powerful editing methods are enabled by GAN latent manipulation, the problem of inverting a GAN is of widespread interest and is the subject of an active line of ongoing research [53, 26, 6, 1, 2, 14, 36, 52, 55, 16].”
GAN Inversion for Out-of-Range Images with Geometric Transformations (2021, methodology): “We compare our method with state-of-the-art methods: Im2StyleGAN [1], StyleGAN2 inversion [16], Pnorm [33], and PSP [22].”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, methodology): “Recent works [2, 61, 73, 75] have demonstrated the ex-”
HyperInverter: Improving StyleGAN Inversion via Hypernetwork (2021, methodology): “Phase I: From image to content code In StyleGAN, W space is empirically proven to have excellent editing ability compared to the popular W space for inversion [44, 59].”
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks (2021, methodology): “Improved-Image2StyleGAN (II2S) (Zhu et al., 2020b) uses P N space to regularize the embeddings for high-quality image reconstruction and image editing.”
Third Time's the Charm? Image and Video Editing with StyleGAN3 (2022, background): “StyleGAN2 [36] features several semantically-rich latent spaces, which have been heavily studied and exploited in the context of image manipulation [4,15,26,60,61,65] and image inversion [1,2,10,25,51,85].”
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022, methodology): “Here, we examine these choices in light of the GAN inversion literature and discover that many core premises (such as a distortion-editability tradeoff (Tov et al., 2021; Zhu et al., 2020b)) also exist in the textual embedding space.”
Scaling up GANs for Text-to-Image Synthesis (2023, unknown): “Notably, StyleGAN-family models [32,34] have shown impressive ability in image synthesis tasks for single-category domains [1, 25, 49, 70, 84].”
Labels4Free: Unsupervised Segmentation using StyleGAN (2021, background): “Some recent works in this domain [5, 6, 16, 18, 49, 3, 36, 2, 37] study the structure of the activation and latent space.”