cGANs with Projection Discriminator

By Takeru Miyato, Masanori Koyama. 2018

827 citations recorded by the source corpus.

We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model. This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors. With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) 1000-class image dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator. We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images. This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator.

Subfields: Biophysics, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Media Technology

In Generative Models

Citations in the corpus

Training Generative Adversarial Networks with Limited Data (2020, background): “The increasingly impressive results of generative adversarial networks (GAN) [14, 32, 31, 5, 19, 20, 21] are fueled by the seemingly unlimited supply of images available online.”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “Recent conditional generators feed the class identifier through a separate embedding network to a large number of layers in the generator [42], while the latent is still provided though the input layer.”
Large Scale GAN Training for High Fidelity Natural Image Synthesis (2018, methodology): “We provide class information to G with class-conditional BatchNorm (Dumoulin et al., 2017; de Vries et al., 2017) and to D with projection (Miyato & Koyama, 2018).”
Spectral Normalization for Generative Adversarial Networks (2018, methodology): “To ensure that the superiority of our method is not limited within our speciﬁc setting, we also compared the performance of SN-GANs against orthonormal regularization on conditional GANs with projection discriminator (Miyato & Koyama, 2018) as well as the standard (unconditional) GANs.”
Diffusion Models Beat GANs on Image Synthesis (2021, background): “This often takes the form of class-conditional normalization statistics [20, 14] as well as discriminators with heads explicitly designed to behave like classifiers p(y|x) [46].”
Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis (2020, methodology): “1 Motivation and Objective R EMARKABLE recent progress has been made on both unconditional and conditional image synthesis [1], [2], [3], [4], [5], [6], [7], [8].”
Object-Centric Image Generation from Layouts (2020, background): “2014) have been at the helm of significant recent advances in image generation (Goodfellow et al. 2014; Radford, Metz, and Chintala 2016; Gulrajani et al. 2017; Miyato and Koyama 2018; Brock, Donahue, and Simonyan 2019).”
A U-Net Based Discriminator for Generative Adversarial Networks (2020, methodology): “Similarly to [5], we provide class information to D with projection [35] to the ch-dimensional channel features of the U-Net encoder and decoder output.”
Improved Precision and Recall Metric for Assessing Generative Models (2019, background): “Due to the large amount of variation within and between classes, generative modeling of ImageNet has proven to be a challenging problem [21, 30, 4].”
Semantic Image Synthesis With Spatially-Adaptive Normalization (2019, methodology): “We test several ResNet-based discriminators used in recent unconditional GANs [1, 36, 39] but observe similar results at the cost of a higher GPU memory requirement.”
Self-Supervised GANs via Auxiliary Rotation Loss (2018, methodology): “We resize the images to 128 × 128 × 3 as done in Miyato and Koyama [21] and Zhang et al. [9].”
Self-Attention Generative Adversarial Networks (2018, methodology): “Similar to [17], SAGAN uses conditional batch normalization in the generator and projection in the discriminator.”
Object-Driven Text-To-Image Synthesis via Adversarial Training (2019, methodology): “We tried combining our discriminators with the spectral normalized projection discriminator [18, 19], but did not see significant performance improvement.”
Classification Accuracy Score for Conditional Generative Models (2019, methodology): “We compare four models: BigGAN, cGAN with Projection Discriminator [30], PixelCNN [19], and PixelIQN [40].”
InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, methodology): “For CIFAR-10, CIFAR-100 and STL-10, we follow settings in [30] by linearly decaying learning rate over 100K generator steps, each taken every 5 discriminator update steps.”
Generative Feature Replay For Class-Incremental Learning (2020, methodology): “We consider conditional GAN with Projection Discriminator [33], which can control the class of generated images.”
Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (2023, methodology): “In contrast, we have adapted Vocos to be a conditional GAN with a projection discriminator (Miyato & Koyama, 2018), and have incorporated adaptive layer normalization (Huang & Belongie, 2017) into the generator.”