GANSpace: Discovering Interpretable GAN Controls

By Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris. 2020

1,034 citations recorded by the source corpus.

This paper describes a simple technique to analyze Generative Adversarial Networks (GANs) and create interpretable controls for image synthesis, such as change of viewpoint, aging, lighting, and time of day. We identify important latent directions based on Principal Components Analysis (PCA) applied either in latent space or feature space. Then, we show that a large number of interpretable controls can be defined by layer-wise perturbation along the principal directions. Moreover, we show that BigGAN can be controlled with layer-wise inputs in a StyleGAN-like manner. We show results on different GANs trained on various datasets, and demonstrate good qualitative matches to edit directions found through earlier supervised approaches.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Diffusion Autoencoders: Toward a Meaningful and Decodable Representation (2021, background): “Besides producing impressive image samples, GANs [15] have been shown to learn meaningful latent spaces [27] with extensive studies on multiple derived spaces [24, 57] and various knobs and controls for conditional human face generation [21, 37, 55].”
StyleGAN-NADA (2021, background): “Such a latent space is conductive for tasks such as image editing and image-to-image translation [3, 13, 30, 33, 36].”
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, methodology): “It may be seen that in GANSpace [13] manipulation is entangled with skin color and lighting, while in InterFace-GAN [41] the identity may change signiﬁcantly (when manipulating Lipstick).”
StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, methodology): “They achieve high image fidelity [Karras et al. 2019, 2020b], fine-grained semantic control [Härkönen et al. 2020; Ling et al. 2021; Wu et al. 2021], and recently alias-free generation enabling realistic animation [Karras et al. 2021].”
Alias-Free Generative Adversarial Networks (2021, background): “While several ways of controlling the generative process have been found [8, 28, 10, 41, 24, 2, 7, 46, 6], the foundations of the synthesis process remain only partially understood.”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “Unlike the previous methods (Abdal et al. 2019; Härkönen et al. 2020; Shen et al. 2019) the semantic edits performed on the latent vectors w forces the resultant vector to remain in the distribution of W space (p(w)). is enables us to do stable sequential edits which, to the best of our…”
Designing an encoder for StyleGAN image manipulation (2021, methodology): “To this end, we follow our inversion method with several existing editing techniques: StyleFlow [3], InterFaceGAN [34], GANSpace [14], and SeFa [35].”
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, methodology): “Recently, numerous methods have shown competence in controlling StyleGAN’s latent space and performing meaningful manipulations in W [17, 35, 36, 13].”
Paint by Word (2021, background): “So recent work has developed unsupervised methods for identifying disentangled interpretable directions in these interior latents [17, 43].”
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation (2020, methodology): “In this section we compare the ability of our approach to achieve disentangled manipulation of visual attributes to that of two state-of-the-art methods, specifically GANSpace [12] and InterFaceGAN [29].”
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields (2020, methodology): “Finally, we discuss how we train our model from raw image collections (Sec.”
Ensembling with Deep Generative Views (2021, methodology): “Alternatively, we also experiment with sampling according to principal directions; these directions were found to correspond well with interpretable controls in GANspace [21].”
Style Intervention: How to Achieve Spatial Disentanglement with Style-based Generators? (2020, methodology): “Unsupervised approaches [9, 11, 22, 26] use classical unsupervised machine learning techniques, e.”
BlobGAN: Spatially Disentangled Scene Representations (2022, background): “Directions of variation naturally emerge in the latent space and can be discovered when guided by geometry/color changes [31], language or attributes [59, 63, 75, 2, 88], cognitive signals [17], or in an unsupervised manner [21, 76, 61].”
Labels4Free: Unsupervised Segmentation using StyleGAN (2021, background): “While state-of-the-art editing frameworks [3, 37, 18] achieve high quality fine grained edits using supervised and unsupervised approaches, background/ foreground aware embeddings and edits still pose a challenge.”
GAN-Supervised Dense Visual Alignment (2021, methodology): “Instead, we parameterize c as a linear combination of the top-N principal directions ofW space [28, 74]:”
Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold (2023, methodology): “Other approaches compute the important semantic directions in the latent space in an unsupervised manner [Härkönen et al. 2020; Shen and Zhou 2020; Zhu et al. 2023].”