Face identity disentanglement via latent space mapping

By Yotam Nitzan, Amit H. Bermano, Yangyan Li, Daniel Cohen-Or. 2020

133 citations recorded by the source corpus.

Learning disentangled representations of data is a fundamental problem in artificial intelligence. Specifically, disentangled latent representations allow generative models to control and compose the disentangled factors in the synthesis process. Current methods, however, require extensive supervision and training, or instead, noticeably compromise quality. In this paper, we present a method that learns how to represent data in a disentangled way, with minimal supervision, manifested solely using available pre-trained networks. Our key insight is to decouple the processes of disentanglement and synthesis, by employing a leading pre-trained unconditional image generator, such as StyleGAN. By learning to map into its latent space, we leverage both its state-of-the-art quality, and its rich and expressive latent space, without the burden of training it. We demonstrate our approach on the complex and high dimensional domain of human heads. We evaluate our method qualitatively and quantitatively, and exhibit its success with de-identification operations and with temporal identity coherency in image sequences. Through extensive experimentation, we show that our method successfully disentangles identity from other facial attributes, surpassing existing methods, even though they require more training and supervision.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, neutral): “Some methods learn to perform image manipulation in an end-to-end fashion, by training a network that encodes a given image into a latent representation of the manipulated image [32, 38, 2].”
Designing an encoder for StyleGAN image manipulation (2021, methodology): “For example, it can be applied to map latent vectors that represent more than one image, or say a combination of two, like the disentangled representation of identity and pose [27], or a blend of two images, to a proper latent code of the target image which is likely to exist in proximity of W .”
One Shot Face Swapping on Megapixels (2021, background): “[35, 5, 28, 38, 36] can be applied to arbitrary identities without additional training procedures.”
A Simple Baseline for StyleGAN Inversion (2021, methodology): “Existing GAN inversion methods can be subdivided into optimization-based [1, 2, 15, 34], forwardbased [35, 33, 6, 16] and hybrid [43, 8, 7] approaches.”
StyleAlign: Analysis and Applications of Aligned StyleGAN Models (2021, background): “…different settings: high-resolution (Wang et al., 2018a), semantic image (Park et al., 2019; Zhu et al., 2020b; Liu et al., 2019b), multi-domain (Choi et al., 2018), multimodal (Zhu et al., 2017b), and using a pre-trained generator (Nitzan et al., 2020; Richardson et al., 2021; Luo et al., 2020).”
Only a matter of style: age transformation using a style-based regression model (2021, methodology): “…manipulations in the latent space of a well-trained GAN generator, either by directly learning a disentangled mapping of an attribute of interest [Nitzan et al. 2020] or by a latent space traversal [Abdal et al. 2020b; Denton et al. 2019; Goetschalckx et al. 2019; Härkönen et al. 2020; Shen et…”
LARGE: Latent-Based Regression through GAN Semantics (2021, methodology): “Many recent works have proposed methods to interpret the semantics encoded in this latent space and apply them to image editing [2, 18, 35, 38, 51, 57, 69].”
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion (2021, methodology): “They can be mainly categorized as optimizationbased [4], [5], [9] and forward-based [10], [6], [7].”