LFS-GAN: Lifelong Few-Shot Image Generation

By Juwon Seo, Jingwen Kang, Gyeong-Moon Park. 2023

22 citations recorded by the source corpus.

We address a challenging lifelong few-shot image generation task for the first time. In this situation, a generative model learns a sequence of tasks using only a few samples per task. Consequently, the learned model encounters both catastrophic forgetting and overfitting problems at a time. Existing studies on lifelong GANs have proposed modulation-based methods to prevent catastrophic forgetting. However, they require considerable additional parameters and cannot generate high-fidelity and diverse images from limited data. On the other hand, the existing few-shot GANs suffer from severe catastrophic forgetting when learning multiple tasks. To alleviate these issues, we propose a framework called Lifelong Few-Shot GAN (LFS-GAN) that can generate high-quality and diverse images in lifelong few-shot image generation task. Our proposed framework learns each task using an efficient task-specific modulator - Learnable Factorized Tensor (LeFT). LeFT is rank-constrained and has a rich representation ability due to its unique reconstruction technique. Furthermore, we propose a novel mode seeking loss to improve the diversity of our model in low-data circumstances. Extensive experiments demonstrate that the proposed LFS-GAN can generate high-fidelity and diverse images without any forgetting and mode collapse in various domains, achieving state-of-the-art in lifelong few-shot image generation task. Surprisingly, we find that our LFS-GAN even outperforms the existing few-shot GANs in the few-shot image generation task. The code is available at Github.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Further to this, some works have focused on other computer vision problems, such as image generation [170], action recognition [171], multi-modal reasoning [172], label-to-image translation [173], and multi-task”
Pre-trained Vision and Language Transformers are Few-Shot Incremental Learners (2024, methodology): “Among many previous approaches for FSCIL [34], dynamic network structure-based methods [29, 33, 38, 40] adjust the network structures themselves during training, while preserving the severe forgetting of the previous knowledge.”