Few-Shot Incremental Learning for Label-to-Image Translation

By Pei-Hung Chen, Yang Zhang, Zejian Li, Lingyun Sun. 2022

11 citations recorded by the source corpus.

Label-to-image translation models generate images from semantic label maps. Existing models depend on large volumes of pixel-level annotated samples. When given new training samples annotated with novel semantic classes, the models should be trained from scratch with both learned and new classes. This hinders their practical applications and motivates us to introduce an incremental learning strategy to the label-to-image translation scenario. In this paper, we introduce a few-shot incremental learning method for label-to-image translation. It learns new classes one by one from a few samples of each class. We propose to adopt semantically-adaptive convolution filters and normalization. When incrementally trained on a novel semantic class, the model only learns a few extra parameters of class-specific modulation. Such design avoids catastrophic forgetting of already-learned semantic classes and enables label-to-image translation of scenes with increasingly rich content. Furthermore, to facilitate few-shot learning, we propose a modulation transfer strategy for better initialization. Extensive experiments show that our method outperforms existing related methods in most cases and achieves zero forgetting.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Lusi Li
6. Song Tian

In Computer Vision and Pattern Recognition

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Lusi Li
6. Song Tian

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “[427]), model decomposition (Hyper-LifelongGAN [488], FILIT [69], GAN-Memory [80]), etc.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Further to this, some works have focused on other computer vision problems, such as image generation [170], action recognition [171], multi-modal reasoning [172], label-to-image translation [173], and multi-task”
• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “Chen et al. (2022) propose a FSIL method for label-to-image translation, which solves this task with semantically-adaptive filters and normalization.”