Semi-Supervised Generation with Cluster-aware Generative Models

By Lars Maaløe, Marco Fraccaro, Ole Winther. 2017

29 citations recorded by the source corpus.

Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning. Many real life data sets contain a small amount of labelled data points, that are typically disregarded when training generative models. We propose the Cluster-aware Generative Model, that uses unlabelled information to infer a latent representation that models the natural clustering of the data, and additional labelled data points to refine this clustering. The generative performances of the model significantly improve when labelled information is exploited, obtaining a log-likelihood of -79.38 nats on permutation invariant MNIST, while also achieving competitive semi-supervised classification accuracies. The model can also be trained fully unsupervised, and still improve the log-likelihood performance with respect to related methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling (2019, methodology): “As shown in [5, 30], we can rewrite the ELBO (eq.”
VAE with a VampPrior (2017, background): “Hierarchical VAE and the inactive stochastic latent variable problem A typical problem encountered during training a VAE is the inactive stochastic units [6, 25].”
DVAE++: Discrete Variational Autoencoders with Overlapping Transformations (2018, background): “Only when z z z has a modest number of con-ﬁgurations (as in semi-supervised learning (Kingma et al., 2014) or semi-supervised generation (Maaløe et al., 2017)) can the gradient of the expectation be decomposed into a summation over conﬁgurations.”
DVAE#: Discrete Variational Autoencoders with Relaxed Boltzmann Priors (2018, background): “i) Exhaustive approaches marginalize all discrete variables [25, 26] and which are not scalable to more than a few discrete variables.”