Lifelong Mixture of Variational Autoencoders

By Fei Ye, Adrian G. Bors. 2021

36 citations recorded by the source corpus.

In this article, we propose an end-to-end lifelong learning mixture of experts. Each expert is implemented by a variational autoencoder (VAE). The experts in the mixture system are jointly trained by maximizing a mixture of individual component evidence lower bounds (MELBO) on the log-likelihood of the given training samples. The mixing coefficients in the mixture model control the contributions of each expert in the global representation. These are sampled from a Dirichlet distribution whose parameters are determined through nonparametric estimation during lifelong learning. The model can learn new tasks fast when these are similar to those previously learned. The proposed lifelong mixture of VAE (L-MVAE) expands its architecture with new components when learning a completely new task. After the training, our model can automatically determine the relevant expert to be used when fed with new data samples. This mechanism benefits both the memory efficiency and the required computational cost as only one expert is used during the inference. The L-MVAE inference model is able to perform interpolations in the joint latent space across the data domains associated with different tasks and is shown to be efficient for disentangled learning representation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fei Ye
2. Adrian G. Bors

In Computer Vision and Pattern Recognition

1. Fei Ye
2. Adrian G. Bors

Citations in the corpus

• Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Another kind of approaches for continual learning focuses on the memory system, which usually employs a small memory buffer [3, 5, 6, 28, 36, 44, 59] for storing some past data or trains a generator to provide the replay samples when learning new tasks [38, 43, 46, 47, 58, 57, 52].”
• Learning an Evolved Mixture Model for Task-Free Continual Learning (2022, background): “branches: regularization-based [8], generative replay mechanism [4, 9] and dynamic expansion approaches [10, 11].”
• Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process (2021, background): “Memory replay approaches would either use a generator [1, 34, 33, 41, 56, 51, 52, 55, 57] or a memory buffer [2, 4, 46] as a replay mechanism generating data, which is statistically consistent to the previously learnt knowledge.”
• Continual Variational Autoencoder Learning via Online Cooperative Memorization (2022, background): “This issue can be solved by storing the information of past samples into the network’s parameters which are then frozen when learning novel tasks [35,64,65,67,68].”
• Lifelong Generative Modelling Using Dynamic Expansion Graph Model (2021, methodology): “We also compare with LIMix (Ye and Bors 2021e) and implement CN-DPM (Lee et al. 2020) with the optimal setting, namely CN-DPM* (See details in Appendix-L.1 from SM1).”