Few-Shot Adaptation of Generative Adversarial Networks

By Esther Robb, Wen-Sheng Chu, Abhishek Kumar, Jia-Bin Huang. 2020

112 citations recorded by the source corpus.

Generative Adversarial Networks (GANs) have shown remarkable performance in image synthesis tasks, but typically require a large number of training samples to achieve high-quality synthesis. This paper proposes a simple and effective method, Few-Shot GAN (FSGAN), for adapting GANs in few-shot settings (less than 100 images). FSGAN repurposes component analysis techniques and learns to adapt the singular values of the pre-trained weights while freezing the corresponding singular vectors. This provides a highly expressive parameter space for adaptation while constraining changes to the pretrained weights. We validate our method in a challenging few-shot setting of 5-100 images in the target domain. We show that our method has significant visual quality gains compared with existing GAN adaptation methods. We report qualitative and quantitative results showing the effectiveness of our method. We additionally highlight a problem for few-shot synthesis in the standard quantitative metric used by data-efficient image synthesis works. Code and additional results are available at this http URL.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

Projected GANs Converge Faster (2021, background): “On small datasets, overﬁtting is a problem that is not detected well by FID and other metrics [64].”
StyleGAN-NADA (2021, background): “Some place restrictions on the space of trainable weights, either during training [25, 34] or by mixing the weights of the fine-tuned model with those of the original [31].”
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, methodology): “While our method does require a pretrained StyleGAN generator, recent works [19, 34] have shown that such a generator can be easily trained with signiﬁcantly fewer examples than required in the past.”
Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis (2021, methodology): “We compute a reconstruction error between all the images from the testing set and The smooth interpolations from the back-tracked latent vectors also suggest little mode-collapse of our G (Radford et al., 2015; Zhao et al., 2020; Robb et al., 2020).”
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning (2023, methodology): “To alleviate overfitting, we draw inspiration from the efficient parameter space in the GAN literature [50] and pro-pose a compact yet efficient parameter space, spectral shift , for diffusion model by only fine-tuning the singular values of the weight matrices of the model.”
Navigating the GAN Parameter Space for Semantic Image Editing (2020, methodology): “Inspired by the recent work on few-shot GAN adaptation [27], we compute the SVD decomposition of the chosen convolutional layer, flattened to a 2D matrix, as in [27].”