Style Intervention: How to Achieve Spatial Disentanglement with Style-based Generators?

By Yunfan Liu, Qi Li, Zhenan Sun, Tien-Ping Tan. 2020

20 citations recorded by the source corpus.

Generative Adversarial Networks (GANs) with style-based generators (e.g. StyleGAN) successfully enable semantic control over image synthesis, and recent studies have also revealed that interpretable image translations could be obtained by modifying the latent code. However, in terms of the low-level image content, traveling in the latent space would lead to `spatially entangled changes' in corresponding images, which is undesirable in many real-world applications where local editing is required. To solve this problem, we analyze properties of the 'style space' and explore the possibility of controlling the local translation with pre-trained style-based generators. Concretely, we propose 'Style Intervention', a lightweight optimization-based algorithm which could adapt to arbitrary input images and render natural translation effects under flexible objectives. We verify the performance of the proposed framework in facial attribute editing on high-resolution images, where both photo-realism and consistency are required. Extensive qualitative results demonstrate the effectiveness of our method, and quantitative measurements also show that the proposed algorithm outperforms state-of-the-art benchmarks in various aspects.

Subfields: Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

1. Daniel Cohen-Or
2. Yuval Alaluf
3. Yotam Nitzan
4. Or Patashnik
5. Omer Tov
6. Chunyuan Li
7. A. B. Impact
8. Yufan Zhou

Citations in the corpus

• Towards Language-Free Training for Text-to-Image Generation (2021, background): “Generator It is shown in recent works [29, 51] that the StyleSpace of StyleGAN2 is a well-disentangled intermediate feature space, whose dimensions are highly independent.”
• Designing an encoder for StyleGAN image manipulation (2021, background): “Last, [24, 40] have considered the so called Style Space , and are able to identify specific components that have a clear effect on spatial regions or on a single attribute.”
• Only a matter of style: age transformation using a style-based regression model (2021, background): “All evaluations are conducted on the CelebA-HQ [Karras et al. 2017] test dataset.”