Mixtures of Variational Autoencoders
By Fei Ye, Adrian G. Bors. 2020
14 citations recorded by the source corpus.
In this paper, we develop a new deep mixture learning framework, aiming to learn underlying complex data structures. Each component in the mixture model is implemented using a Variational Autoencoder (VAE). VAE is a well known deep learning model which models a latent space data representation on a variational manifold. The mixing parameters are estimated from a Dirichlet distribution modelled by each encoder. In order to train this mixture model, named M-VAE, we derive a mixture evidence lower bound on the sample log-likelihood, which is optimized in order to jointly estimate all mixture components. We further propose to use the d-variables Hilbert-Schmidt Independence Criterion (dHSIC) as a regularization criterion in order to enforce the independence among the encoders’ distributions. This criterion encourages the proposed mixture components to learn different data distributions and represent them in the latent space. During the experiments with the proposed M-VAE model we observe that it can be used for discovering disentangled data representations which can not be achieved with a single VAE.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Statistical and Nonlinear Physics
Citations in the corpus
- Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Due to its robust generation and inference mechanisms, VAEs have been widely used in many applications [48, 49, 50, 51, 54, 61].”
- Continual Variational Autoencoder Learning via Online Cooperative Memorization (2022, methodology): “VAEs have been widely used in image synthesis [60,62], semi-supervised learning [1,63] and for image-to-image translation [38].”
- Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process (2021, methodology): “Each pθj (x | z) is implemented as a Gaussian distribution N (gθj (z),Σ) with Σ considered as a diagonal matrix, and gθj represents the deterministic mapping that maps z into the mean of pθj (x | z), implemented as a deep learning network [53, 54].”
- Lifelong Generative Modelling Using Dynamic Expansion Graph Model (2021, methodology): “A major limitation for GRbased models is their inability of learning a long sequence of data domain due to the fixed model capacity while having to retrain the generator frequently (Ye and Bors 2020a).”
- Deep Mixture Generative Autoencoders (2021, methodology): “A mixture of VAEs defined in a fixed architecture was proposed in [36].”
- Learning an Evolved Mixture Model for Task-Free Continual Learning (2022, background): “Another approach for TFCL is based on the expansion of the network architecture or by adding new components to a mixture model [19, 20, 21].”