StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation

By Zongze Wu, Dani Lischinski, Eli Shechtman. 2020

540 citations recorded by the source corpus.

We explore and analyze the latent style space of Style-GAN2, a state-of-the-art architecture for image generation, using models pretrained on several different datasets. We first show that StyleSpace, the space of channel-wise style parameters, is significantly more disentangled than the other intermediate latent spaces explored by previous works. Next, we describe a method for discovering a large collection of style channels, each of which is shown to control a distinct visual attribute in a highly localized and dis-entangled manner. Third, we propose a simple method for identifying style channels that control a specific attribute, using a pretrained classifier or a small number of example images. Manipulation of visual attributes via these StyleSpace controls is shown to be better disentangled than via those proposed in previous works. To show this, we make use of a newly proposed Attribute Dependency metric. Finally, we demonstrate the applicability of StyleSpace controls to the manipulation of real images. Our findings pave the way to semantically meaningful and well-disentangled image manipulations via simple and intuitive interfaces.

Subfields: Cognitive Neuroscience, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Towards Language-Free Training for Text-to-Image Generation (2021, background): “Generator It is shown in recent works [29, 51] that the StyleSpace of StyleGAN2 is a well-disentangled intermediate feature space, whose dimensions are highly independent.”
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation (2021, methodology): “There exists specialized techniques for this task [3,37,44,57], but here we aim to showcase the quality and applicability of our latent space by using the simplest linear operation.”
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, methodology): “Specifically, in addition to the results shown previously, where the methods were applied in W+, here we apply these methods in the StyleSpace, S [10].”
StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, methodology): “2020a] and as it does not employ an intermediate latent space which is essential for GAN-based image editing [Abdal et al. 2021; Collins et al. 2020; Patashnik et al. 2021; Wu et al. 2021].”
Paint by Word (2021, background): “So recent work has developed unsupervised methods for identifying disentangled interpretable directions in these interior latents [17, 43].”
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data (2023, background): “CLIP [64] learns to map images and text captions into a shared embedding space, proving useful for many (often zero-shot) tasks [40, 65, 86].”
Lafite2: Few-shot Text-to-Image Generation (2022, methodology): “The text features will be updated by (2) and injected into the Style Space [37] of StyleGAN2.”
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks (2021, background): “…2021; Tritrong et al., 2021; Abdal et al., 2021a; Collins et al., 2020; Bielski & Favaro, 2019), extracting 3D models of the objects (Pan et al., 2021; Chan et al., 2020) and other semantic image editing applications (Zhu et al., 2021; Tan et al., 2020; Wu et al., 2020; Patashnik et al., 2021).”
Zero-shot Image-to-Image Translation (2023, background): “2021], explore different latent spaces [Abdal et al. 2019, 2020; Wu et al. 2021], invert into multiple layers [Gu et al.”
Only a matter of style: age transformation using a style-based regression model (2021, methodology): “Other works [Abdal et al. 2020b; Härkönen et al. 2020; Liu et al. 2020; Shen et al. 2020; Wu et al. 2020] have approached the age transformation task by exploring the semantics of the latent space of a well-trained GAN, such as StyleGAN [Karras et al.”
Explaining in Style: Training a GAN to explain a classifier in StyleSpace (2021, methodology): “But more important to our goal is the observation recently made by [35] that StyleGAN2 tends to inherently contain a disentangled StyleSpace space, which can be used to extract individual attributes.”
Cross-Domain and Disentangled Face Manipulation With 3D Guidance (2021, methodology): “The space spanned by these style parameters is referred to as the style space [52], denoted as S.”
Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs (2021, methodology): “We adopted the StyleGAN2 to evaluate Local Basis because of its disentanglement property observed in [4, 15, 21].”
High-fidelity GAN Inversion with Padding Space (2022, background): “Existing studies on GAN interpretation have affirmed that, pre-trained GAN models own great potential in a range of downstream applications, such as object classification [9, 45], semantic segmentation [49], video generation [23], and image editing [4, 11, 13, 20, 28, 31, 38, 39, 42, 46, 50].”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, background): “We hypothesize that modifying these layers mainly alters the pixel-wise texture and color [67], changes that do not translate well under global edits such as pose (see the supplementary materials for examples).”
Labels4Free: Unsupervised Segmentation using StyleGAN (2021, methodology): “While originally conjectured that GANs are merely great at memorizing the training data, recent work in GAN-based image editing [18, 3, 36, 42] demonstrates that GANs learn non-trivial semantic information about a class of objects, e.”
Low-Rank Subspaces in GANs (2021, methodology): “Zoom in for details. opening eyes, in which we could ﬁnd that our methods could achieve more photo-realistic results compared with StyleSpace [37].”
BlobGAN: Spatially Disentangled Scene Representations (2022, background): “Directions of variation naturally emerge in the latent space and can be discovered when guided by geometry/color changes [31], language or attributes [59, 63, 75, 2, 88], cognitive signals [17], or in an unsupervised manner [21, 76, 61].”