Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs

By Sangwoo Mo, Minsu Cho, Jinwoo Shin. 2020

235 citations recorded by the source corpus.

Generative adversarial networks (GANs) have shown outstanding performance on a wide range of problems in computer vision, graphics, and machine learning, but often require numerous training data and heavy computational resources. To tackle this issue, several methods introduce a transfer learning technique in GAN training. They, however, are either prone to overfitting or limited to learning small distribution shifts. In this paper, we show that simple fine-tuning of GANs with frozen lower layers of the discriminator performs surprisingly well. This simple baseline, FreezeD, significantly outperforms previous techniques used in both unconditional and conditional GANs. We demonstrate the consistent effect using StyleGAN and SNGAN-projection architectures on several datasets of Animal Face, Anime Face, Oxford Flower, CUB-200-2011, and Caltech-256 datasets. The code and results are available at https://github.com/sangwoomo/FreezeD.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

StyleGAN-NADA (2021, background): “Some place restrictions on the space of trainable weights, either during training [25, 34] or by mixing the weights of the fine-tuned model with those of the original [31].”
Differentiable Augmentation for Data-Efficient GAN Training (2020, methodology): “A.3 Few-Shot Generation We compare our method to transfer learning algorithms using the FreezeD’s codebase§ based on the pre-trained StyleGAN model from the FFHQ dataset [15] at 256×256 resolution.”
Few-shot Image Generation via Cross-domain Correspondence (2021, methodology): “Figure 4: Comparison of our method compared to EWC [5] and FreezeD [6] in 1-shot, 5-shot and 10-shot setting.”
Lafite2: Few-shot Text-to-Image Generation (2022, methodology): “Freeze-D [20] is used during ﬁne-tuning for L AFITE 2 GAN .”
Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis (2021, methodology): “Training from scratch vs. ﬁne-tuning : Fine-tuning from a pre-trained GAN (Mo et al., 2020; Noguchi & Harada, 2019; Wang et al., 2020) has been the go-to method for the image generation task on datasets with few samples.”
StyleAlign: Analysis and Applications of Aligned StyleGAN Models (2021, background): “…advantages of ﬁne-tuning are enabling few-shot tasks such as classiﬁcation (Chen et al., 2019) and unconditional generation (Wang et al., 2018b; Mo et al., 2020; Wang et al., 2020; Li et al., 2020; Ojha et al., 2021), improved performance in a wide variety of tasks (Devlin et al., 2018;…”
LFS-GAN: Lifelong Few-Shot Image Generation (2023, background): “FreezeD [34] fine-tunes a pre-trained GAN by freezing the earlier layers of the discriminator.”
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks (2021, methodology): “A discriminator related approach, FreezeD (Mo et al., 2020), freezes lower layers of the discriminator to achieve domain adaptation.”
On Leveraging Pretrained GANs for Generation with Limited Data (2020, methodology): “…more due to more trainable parameters; as our method has a much higher training efficiency, a false impression may potentially arise (Cong et al., 2019; Mo et al., 2020); for clarity, see the significantly improved best performance of our method and the more gentle ending slope of its FID curves.”
Few-Shot Adaptation of Generative Adversarial Networks (2020, methodology): “The standard evaluation practice used in prior low-shot GAN adaptation work (Wang et al., 2018; Noguchi & Harada, 2019; Mo et al., 2020) is to estimate FID (Heusel et al.”
Regularizing Generative Adversarial Networks under Limited Data (2021, methodology): “model [15], modifying the batch statics [12], using an additional mining network [14], and fine-tuning parts of the GAN model [11].”
Few-Shot Incremental Learning for Label-to-Image Translation (2022, methodology): “In the fine-tuning process, the whole generator is used to learn new tasks, and the decoder of discriminator is fixed [34].”
GAN-Supervised Dense Visual Alignment (2021, methodology): “For CUB, we first pre-train a StyleGAN2 with ADA [40] on the NABirds dataset [78] and fine-tune it with FreezeD [60] on the training split of CUB, using the same image pre-processing and dataset splits as ACSM [45] for a fair comparison.”
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning (2023, background): “This approach is inspired by prior work in GAN adaptation showing that constraining the space of trainable parameters can lead to improved performance on target domain [48, 36, 41, 64].”