StyleAlign: Analysis and Applications of Aligned StyleGAN Models

By Zongze Wu, Yotam Nitzan, Eli Shechtman, Dani Lischinski. 2021

65 citations recorded by the source corpus.

In this paper, we perform an in-depth study of the properties and applications of aligned generative models. We refer to two models as aligned if they share the same architecture, and one of them (the child) is obtained from the other (the parent) via fine-tuning to another domain, a common practice in transfer learning. Several works already utilize some basic properties of aligned StyleGAN models to perform image-to-image translation. Here, we perform the first detailed exploration of model alignment, also focusing on StyleGAN. First, we empirically analyze aligned models and provide answers to important questions regarding their nature. In particular, we find that the child model's latent spaces are semantically aligned with those of the parent, inheriting incredibly rich semantics, even for distant data domains such as human faces and churches. Second, equipped with this better understanding, we leverage aligned models to solve a diverse set of tasks. In addition to image translation, we demonstrate fully automatic cross-domain image morphing. We further show that zero-shot vision tasks may be performed in the child domain, while relying exclusively on supervision in the parent domain. We demonstrate qualitatively and quantitatively that our approach yields state-of-the-art results, while requiring only simple fine-tuning and inversion.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yuval Alaluf
2. Rinon Gal
3. Omer Tov
4. Amit H. Bermano
5. Ron Mokady
6. Yonatan Shafir
7. Roy Kapon
8. Guy Tevet

In Generative Models

1. Yuval Alaluf
2. Rinon Gal
3. Omer Tov
4. Amit H. Bermano
5. Ron Mokady
6. Yonatan Shafir
7. Roy Kapon
8. Guy Tevet

Citations in the corpus

• StyleGAN-NADA (2021, neutral): “Second, because prior and concurrent methods successfully employed the natural alignment of fine-tuned generators for downstream applications [38, 46, 53, 57].”
• Human Motion Diffusion as a Generative Prior (2023, background): “We argue that this idea can be generalized to any two "aligned" (see definition in [Wu et al. 2021]) diffusion models 𝐺 and 𝐺 that are conditioned on 𝑐 𝑎 and 𝑐 𝑏 respectively.”
• HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, background): “This process maintains a correspondence between semantic attributes in the two latent spaces, allowing translation between domains [68].”