Learning latent representations across multiple data domains using Lifelong VAEGAN

By Fei Ye, Adrian G. Bors. 2020

83 citations recorded by the source corpus.

The problem of catastrophic forgetting occurs in deep learning models trained on multiple databases in a sequential manner. Recently, generative replay mechanisms (GRM), have been proposed to reproduce previously learned knowledge aiming to reduce the forgetting. However, such approaches lack an appropriate inference model and therefore can not provide latent representations of data. In this paper, we propose a novel lifelong learning approach, namely the Lifelong VAEGAN (L-VAEGAN), which not only induces a powerful generative replay network but also learns meaningful latent representations, benefiting representation learning. L-VAEGAN can allow to automatically embed the information associated with different domains into several clusters in the latent space, while also capturing semantically meaningful shared latent variables, across different data domains. The proposed model supports many downstream tasks that traditional generative replay methods can not, including interpolation and inference across different data domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fei Ye
2. Adrian G. Bors
3. Hang Su
4. Jun Zhu
5. Jun Liu
6. Hossein Rahmani
7. Liyuan Wang
8. Xingxing Zhang

In Computer Vision and Pattern Recognition

1. Fei Ye
2. Adrian G. Bors
3. Jun Liu
4. Hossein Rahmani
5. Hang Su
6. Jun Zhu
7. Liyuan Wang
8. Senthil Purushwalkam

In Generative Models

1. Fei Ye
2. Adrian G. Bors
3. Jian Zhang
4. Liyuan Wang
5. Jun Liu
6. Chen Zhao
7. Jun Zhu
8. Rui Gao

In Representation & Self-Supervised Learning

1. Fei Ye
2. Adrian G. Bors
3. Senthil Purushwalkam
4. Liyuan Wang
5. Jian Zhang
6. Jun Liu
7. Chen Zhao
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “L-VAEGAN [472] instead employs a hybrid model for both high-quality”
• Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Recent attempts have provided the theoretical analysis for CL from different perspectives including the risk bound [46, 51], NP-hard problem [17], Teacher-Student framework [23, 58] and game theory [37].”
• The Challenges of Continuous Self-Supervised Learning (2022, background): “However, recent works [1, 63, 66, 82] have attempted to address a sub-problem of ours, i.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “A few recent works [140, 141] have also proposed to generate both previous data instances and their latent representations to further consolidate previously learned knowledge.”
• Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…et al., 2019; Abati et al., 2020; Liu, Su, Liu, Schiele, & Sun, 2020; Titsias, Schwarz, de G. Matthews, Pascanu, & Teh, 2020; von Oswald, Henning, Sacramento, & Grewe, 2020; Ye & Bors, 2020) or midlevel CNN feature representations (van de Ven et al., 2020; Lao, Jiang, Havaei, & Bengio, 2020).”
• InfoVAEGAN: Learning Joint Interpretable Representations by Information Maximization and Maximum Likelihood (2021, background): “Lately, the likelihood estimation as a regularization term was shown to stabilize adversarial distribution matching [3].”
• Task-agnostic Continual Learning with Hybrid Probabilistic Models (2021, unknown): “Ye and Bors [2020] embed the information associated with different domains into several clusters.”
• Lifelong Mixture of Variational Autoencoders (2021, methodology): “Recently, there have been some attempts to learn cross-domain representations under lifelong learning by introducing an environment-dependent mask that specifies a subset of generative factors [16], by proposing a teacher–student lifelong learning framework [15], [36] or a hybrid model [37] of GANs ”
• R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning (2022, background): “Some prior works [24,3,9,30,28] train a large generator simultaneously with the training of the solver model, which helps the solver”
• DDGR: Continual Learning with Deep Diffusion-based Generative Replay (2023, background): “…a generator that employs a brain-inspired dual-memory system to instruct the classifier training; Wu et al. (2018) propose MeRGANs, which can generate images without forgetting; Ye & Bors (2020) propose L-VAEGAN, which induces generative replay samples and learns informative latent representations.”
• Lifelong Generative Modelling Using Dynamic Expansion Graph Model (2021, methodology): “A major limitation for GRbased models is their inability of learning a long sequence of data domain due to the fixed model capacity while having to retrain the generator frequently (Ye and Bors 2020a).”
• Learning joint latent representations based on information maximization (2021, methodology): “The Lifelong VAEGAN was used for learning sequentially several databases in [30].”
• Learning an Evolved Mixture Model for Task-Free Continual Learning (2022, background): “However, deep learning systems would usually perform well on individual tasks [2, 3] but suffer from dramatic performance loss when training on several different tasks sequentially [4, 5, 6].”