Zero-Shot Learning via Class-Conditioned Deep Generative Models

By Wenlin Wang, Yunchen Pu, Vinay Kumar Verma, Kai Fan, Yizhe Zhang, Changyou Chen, Piyush Rai, Lawrence Carin. 2017

154 citations recorded by the source corpus.

We present a deep generative model for Zero-Shot Learning (ZSL). Unlike most existing methods for this problem, that represent each class as a point (via a semantic embedding), we represent each seen/unseen class using a class-specific latent-space distribution, conditioned on class attributes. We use these latent-space distributions as a prior for a supervised variational autoencoder (VAE), which also facilitates learning highly discriminative feature representations for the inputs. The entire framework is learned end-to-end using only the seen-class training data. At test time, the label for an unseen-class test input is the class that maximizes the VAE lower bound. We further extend the model to a (i) semi-supervised/transductive setting by leveraging unlabeled unseen-class data via an unsupervised learning module, and (ii) few-shot learning where we also have a small number of labeled inputs from the unseen classes. We compare our model with several state-of-the-art methods through a comprehensive set of experiments on a variety of benchmark data sets.

Subfields: Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Meta & Continual Learning

Citations in the corpus

Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, background): “Adversarial learning [6, 26] has been shown to be successful in dialog generation, translation, visual captioning and other natural language generation tasks [7, 12, 27, 28, 29, 30, 31, 32, 33, 34].”
Deconvolutional Latent-Variable Model for Text Sequence Matching (2017, background): “So motivated, we propose to infer the embedding for each sentence with deep generative models, due to their ability to make effective use of unlabeled data and learn abstract features from complex data (Kingma et al. 2014; Yang et al. 2017; Pu et al. 2016; Wang et al. 2017).”
Large-Scale Few-Shot Learning: Knowledge Transfer With Class Hierarchy (2019, methodology): “Furthermore, our feature learning model can be easily extended to the closely related zero-shot learning (ZSL) problem [32, 17, 29, 31, 14].”
Topic Compositional Neural Language Model (2017, background): “Variational inference has been successfully applied in a variety of applications (Pu et al., 2016; Wang et al., 2017; Chen et al., 2017).”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, background): “Recently, generative models [7, 53, 47, 16, 49] have shown promising results for both ZSL and GZSL setups.”
A Zero-Shot Framework for Sketch-based Image Retrieval (2018, background): “Zero-shot learning in Image Classiﬁcation [25,26,27,28] refers to learning to recognize images of novel classes although no examples from these classes are present in the training set.”
Towards Better Representations with Deep/Bayesian Learning (2018, background): “…et al., 2017; Fan et al., 2016a; Pu et al., 2016a,b; Chen et al., 2018b; Li et al., 2018a; Pu et al., 2017b; Chen et al., 2018a; Xu et al., 2017; Wang et al., 2017b; Pu et al., 2018b; Yuan et al., 2017, 2018; Chapfuwa et al., 2018), sigmoid belief networks (Gan et al., 2015b,c,a; Zhang et al.,…”
Generalized Zero-Shot Learning via Synthesized Examples (2017, background): “Generative models [30, 10, 3, 31] are promising in this setting.”
A Generative Model for Zero Shot Learning Using Conditional Variational Autoencoders (2017, unknown): “Hence, non linear compatibility learning methods have also been proposed[40, 42].”
Domain-Invariant Projection Learning for Zero-Shot Recognition (2018, background): “More importantly, although our model only takes a simple linear formulation, it is clearly shown to outperform existing nonlinear autoencoder-based ZSL models (including transductive ones) [59, 38] (see Table 2).”
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-Based Image Retrieval (2019, background): “Although, most of the deep neural network models in this domain are trained using a discriminative loss function, a few generative models also exist [52, 55, 8] that are used as a data augmentation mechanism.”
Gradient Matching Generative Networks for Zero-Shot Learning (2019, methodology): “[44] extends this notion with trainable class conditional latent spaces.”