Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation

By Mengyao Zhai, Lei Chen, Greg Mori. 2021

46 citations recorded by the source corpus.

Deep neural networks are susceptible to catastrophic forgetting: when encountering a new task, they can only remember the new task and fail to preserve its ability to accomplish previously learned tasks. In this paper, we study the problem of lifelong learning for generative models and propose a novel and generic continual learning framework Hyper-LifelongGAN which is more scalable compared with state-of-the-art approaches. Given a sequence of tasks, the conventional convolutional filters are factorized into the dynamic base filters which are generated using task specific filter generators, and deterministic weight matrix which linearly combines the base filters and is shared across different tasks. Moreover, the shared weight matrix is multiplied by task specific coefficients to introduce more flexibility in combining task specific base filters differently for different tasks. Attributed to the novel architecture, the proposed method can preserve or even improve the generation quality at a low cost of parameters. We validate Hyper-LifelongGAN on diverse image-conditioned generation tasks, extensive ablation studies and comparisons with state-of-the-art models are carried out to show that the proposed approach can address catastrophic forgetting effectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Meta & Continual Learning

Citations in the corpus

Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Aside from image classification, a few recent works [155, 156] have focused on the conditional image generation problem.”
Deep Class-Incremental Learning: A Survey (2023, background): “the generative model also suffers the catastrophic forgetting problem [126], [127], [128], which in turn increases the forgetting in incremental model training.”
Few-Shot Incremental Learning for Label-to-Image Translation (2022, background): “[56] and the number of Task 20 is calculated accordingly.”
Continually Learning Self-Supervised Representations with Projected Functional Regularization (2021, methodology): “Replaybased methods save a small amount of data from previously seen tasks [4, 9] or generate synthetic data with a generative model [62, 70].”