Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation

By Mengyao Zhai, Lei Chen, Jiawei He, Megha Nawhal, Frederick Tung, Greg Mori. 2021

32 citations recorded by the source corpus.

Humans accumulate knowledge in a lifelong fashion. Modern deep neural networks, on the other hand, are susceptible to catastrophic forgetting: when adapted to perform new tasks, they often fail to preserve their performance on previously learned tasks. Given a sequence of tasks, a naive approach addressing catastrophic forgetting is to train a separate standalone model for each task, which scales the total number of parameters drastically without efficiently utilizing previous models. In contrast, we propose a parameter efficient framework, Piggyback GAN, which learns the current task by building a set of convolutional and deconvolutional filters that are factorized into filters of the models trained on previous tasks. For the current task, our model achieves high generation quality on par with a standalone model at a lower number of parameters. For previous tasks, our model can also preserve generation ar X iv :2 10 4. 11 93 9v 1 [ cs .C V ] 2 4 A pr 2 02 1

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Zhu
2. Hang Su
3. Liyuan Wang
4. Xingxing Zhang
5. Greg Mori
6. Mengyao Zhai
7. Lei Chen
8. Pei-Hung Chen

In Computer Vision and Pattern Recognition

1. Jun Zhu
2. Hang Su
3. Liyuan Wang
4. Xingxing Zhang
5. Greg Mori
6. Mengyao Zhai
7. Lei Chen
8. Yang Zhang

In Generative Models

1. Greg Mori
2. Mengyao Zhai
3. Lei Chen
4. Liyuan Wang
5. Jun Zhu
6. Hang Su
7. Ying Zhao
8. Lingyun Sun

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Greg Mori
4. Hang Su
5. Jun Zhu
6. Mengyao Zhai
7. Lei Chen
8. Pei-Hung Chen

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “ing of conditional GANs, such as by weight regularization (ORDisCo [428], [374]), generative replay (MeRGANs [452], L-VAEGAN [472]), knowledge distillation (LifelongGAN [489], Hyper-LifelongGAN [488]), parameter allocation (PiggybackGAN [487], DGMa/DGMw [317], TMNs”
• Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, background): “…on task specific information to route through the architecture (Mundt et al., 2020), while other studies (Rusu et al., 2016; Aljundi et al., 2017; Zhai et al., 2020; Madotto et al., 2021; Ke et al., 2021; Geng et al., 2021; Zhao et al., 2022b) dynamically grow the architecture in the LL training…”
• Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation (2021, methodology): “More generic continual learning frameworks [41, 40] have been proposed enabling lifelong learning of image-conditioned generation tasks.”
• Few-Shot Incremental Learning for Label-to-Image Translation (2022, methodology): “LifelongGAN and PiggybackGAN are trained with our twophase pipeline.”