StarGAN v2: Diverse Image Synthesis for Multiple Domains

By Yunjey Choi, Youngjung Uh, Jaejun Yoo, Jung-Woo Ha. 2019

2,118 citations recorded by the source corpus.

A good image-to-image translation model should learn a mapping between different visual domains while satisfying the following properties: 1) diversity of generated images and 2) scalability over multiple domains. Existing methods address either of the issues, having limited diversity or multiple models for all domains. We propose StarGAN v2, a single framework that tackles both and shows significantly improved results over the baselines. Experiments on CelebA-HQ and a new animal faces dataset (AFHQ) validate our superiority in terms of visual quality, diversity, and scalability. To better assess image-to-image translation models, we release AFHQ, high-quality animal faces with large inter- and intra-domain differences. The code, pretrained models, and dataset are available at https://github.com/clovaai/stargan-v2.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Training Generative Adversarial Networks with Limited Data (2020, methodology): “Figure 16: Uncurated 512 × 512 results generated for AFHQ W ILD [6] (4738 images) with and without ADA, along with real images from the training set.”
Projected GANs Converge Faster (2021, methodology): “Lastly, we evaluate on AFHQ-Cat, -Dog and -Wild at 512(2) [5].”
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, background): “Here, we evaluate a pretrained StyleGAN2 generator [38] trained on FFHQ (1024), MetFaces (1024), and AFHQ (512) dataset images, and calculate FID with 50,000 images.”
A Note on Data Biases in Generative Models (2020, background): “1 which demonstrates even more drastic improvements when transitioning from the AnimalFaces [12] dataset to its corresponding high-quality version, AnimalFacesHQ [9].”
StyleGAN-NADA (2021, methodology): “We use existing editing techniques and show that latent-space directions do indeed AFHQ-Dog [11].”
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, methodology): “Here we use StyleGAN-ada [5] pretrained on AFHQ wild [2], which contains wolves, lions, tigers and foxes.”
Alias-Free Generative Adversarial Networks (2021, methodology): “We used an updated version of the AFHQ dataset [14] where the resampling ﬁltering has been improved.”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, background): “While the resultant conditional GANs (Choi et al. 2019; Park et al. 2019; Xiao et al. 2018) provide a level of (semantic) aribute control, the resultant image qualities can be blurry and fail to reach the quality produced by uncontrolled GANs like StyleGAN.”
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, background): “Since then, their work has been extended for many scenarios: high-resolution synthesis [38], unsupervised learning [27, 43, 22, 26], multi-modal image synthesis [44, 14, 6], and conditional image synthesis [30, 23, 28, 45, 5].”
ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement (2021, methodology): “[13], we trained e4e on the AFHQ [1] Wild dataset using the ofﬁcial implementation and default e4e hyper-parameters.”
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs (2020, background): “, image generation [5, 20, 21], image-to-image translation [28, 33, 8], and video-to-video synthesis [43, 3, 6].”
Diffusion Models already have a Semantic Latent Space (2022, methodology): “Extensive experiments demonstrate that our method is generally applicable to various architectures (DDPM++, iDDPM, and ADM) and datasets (CelebA-HQ, AFHQ-dog, LSUN-church, LSUN-bedroom, and M ET F ACES ).”
Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs (2020, background): “Generative Adversarial Nets (GANs) [14] have achieved remarkable success in many applications, such as image synthesis [20, 19, 21] and image translation [8, 10, 9, 33].”
Few-shot Image Generation with Elastic Weight Consolidation (2020, methodology): “To validate this, we select the FFHQ [16] face dataset as the source domain and several target domains for adaptation according to their dissimilarity with FFHQ: the CelebA-Female face [24], the emoji face [11], the cat face [3], and the color pencil landscape [29].”
StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion (2021, methodology): “The discriminator D in [10] has shared layers that learns the common features between real and fake samples in all domains, followed by a domain-specific binary classifier that classifies whether a sample is real in each domain y ∈ Y .”
On Aliased Resizing and Surprising Subtleties in GAN Evaluation (2022, background): “Here, we evaluate a pretrained StyleGAN2 generator [34] trained on FFHQ (1024), MetFaces (1024), and AFHQ (512) dataset images, and compute the FID with 50,000 images.”
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion (2023, methodology): “Figure 6: Comparison of score-based models (EDM), distillation models (CM), and CTM with various sampling methods and NFE trained on AFHQ-cat (Choi et al., 2020) 256× 256.”
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (2021, methodology): “[17], and extended to unsupervised [15, 58], few-shot [24], and multi-domain image translations [7].”