On Leveraging Pretrained GANs for Limited-Data Generation

By Miaoyun Zhao, Yulai Cong, Lawrence Carin. 2020

22 citations recorded by the source corpus.

Recent work has shown generative adversarial networks (GANs) can generate highly realistic images, that are often indistinguishable (by humans) from real images. Most images so generated are not contained in the training dataset, suggesting potential for augmenting training sets with GAN-generated data. While this scenario is of particular relevance when there are limited data available, there is still the issue of training the GAN itself based on that limited data. To facilitate this, we leverage existing GAN models pretrained on large-scale datasets (like ImageNet) to introduce additional knowledge (which may not exist within the limited data), following the concept of transfer learning. Demonstrated by natural-image generation, we reveal that low-level filters (those close to observations) of both the generator and discriminator of pretrained GANs can be transferred to facilitate generation in a perceptually-distinct target domain with limited training data. To further adapt the transferred filters to the target domain, we propose adaptive filter modulation (AdaFM). An extensive set of experiments is presented to demonstrate the effectiveness of the proposed techniques on generation with limited data.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Changyou Chen
2. Artem Babenko
3. Jianfeng Gao
4. Yang Zhao
5. Chunyuan Li
6. Andrey Voynov
7. T. Grigoryev
8. Ping Yu

Citations in the corpus

• GAN Memory with No Forgetting (2020, methodology): “Different from FiLM modulating latent features, another technique named adaptive filter modulation (AdaFM) modulates source convolutional (Conv) filters to manipulate its “style” to deliver a boosted transfer performance [95].”
• Feature Quantization Improves GAN Training (2020, background): “(iii) In particular, the distribution of fake samples shifts as the generator changes during training, making the classification task for discriminator evolve over time (Chen et al., 2019; Liang et al., 2018; Zhao et al., 2020; Cong et al., 2020).”
• When, Why, and Which Pretrained GANs Are Useful? (2022, background): “A line of works (Wang et al., 2018b; Noguchi & Harada, 2019; Zhao et al., 2020; Mo et al., 2020; Wang et al., 2020; Li et al., 2020) investigate different methods to transfer GANs to new datasets and report significant advantages compared to training from scratch both in terms of generative quality ”