Disentangling in Latent Space by Harnessing a Pretrained Generator
By Yotam Nitzan, Amit H. Bermano, Yangyan Li, Daniel Cohen-Or. 2020
16 citations recorded by the source corpus.
Learning disentangled representations of data is a fundamental problem in artificial intelligence. Specifically, disentangled latent representations allow generative models to control and compose the disentangled factors in the synthesis process. Current methods, however, require extensive supervision and training, or instead, noticeably compromise quality. In this paper, we present a method that learn show to represent data in a disentangled way, with minimal supervision, manifested solely using available pre-trained networks. Our key insight is to decouple the processes of disentanglement and synthesis, by employing a leading pre-trained unconditional image generator, such as StyleGAN. By learning to map into its latent space, we leverage both its state-of-the-art quality generative power, and its rich and expressive latent space, without the burden of training it.We demonstrate our approach on the complex and high dimensional domain of human heads. We evaluate our method qualitatively and quantitatively, and exhibit its success with de-identification operations and with temporal identity coherency in image sequences. Through this extensive experimentation, we show that our method successfully disentangles identity from other facial attributes, surpassing existing methods, even though they require more training and supervision.
Subfields: Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Representation & Self-Supervised Learning
- Daniel Cohen-Or
- Or Patashnik
- Yuval Alaluf
- Yotam Nitzan
- Eli Shechtman
- Dani Lischinski
- Zongze Wu
- Bolei Zhou
Citations in the corpus
- GAN Inversion: A Survey (2021, methodology): “This characteristic is also named multi-dimensional [54] or conditional editing [21] in other GAN inversion approaches.”
- StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, neutral): “Nevertheless, we believe it will be useful for the reader to judge our work in competition with these recent papers (Härkönen et al. 2020; Nitzan et al. 2020; Tewari et al. 2020a), because they provide beer results than other work.”
- Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, methodology): “These methods follow an “ invert first, edit later ” approach, where one first inverts an image into StyleGAN’s latent space and then ed-its the latent code in a semantically meaningful manner to obtain a new code that is then used by StyleGAN to generate the output image.”
- StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation (2020, methodology): “[21] use the generator as a fixed decoder, and facilitate disentanglement by training an encoder for identity and another encoder for pose.”
- One Shot Face Swapping on Megapixels (2021, methodology): “Secondly, to modify face representations and resolve the problem of previous latent code manipulation methods [43, 17, 48, 51, 50, 7, 37, 3] that only one attribute can be modified once a time, a novel swapping module, Face Transfer Module (FTM), is proposed to control multiple attributes synchronou”