A Generative Model for Zero Shot Learning Using Conditional Variational Autoencoders

By Ashish Mishra, Shiva Krishna Reddy Malay, Anurag Mittal, Hema A. Murthy. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017

328 citations recorded by the source corpus.

Zero shot learning in Image Classification refers to the setting where images from some novel classes are absent in the training data but other information such as natural language descriptions or attribute vectors of the classes are available. This setting is important in the real world since one may not be able to obtain images of all the possible classes at training. While previous approaches have tried to model the relationship between the class attribute space and the image space via some kind of a transfer function in order to model the image space correspondingly to an unseen class, we take a different approach and try to generate the samples from the given attributes, using a conditional variational autoencoder, and use the generated samples for classification of the unseen classes. By extensive testing on four benchmark datasets, we show that our model outperforms the state of the art, particularly in the more realistic generalized setting, where the training classes can also appear at the test time along with the novel classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/c3124b0491d479e8a869c61f059ffa08dad49362

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A Variational Autoencoder with Deep Embedding Model for Generalized Zero-Shot Learning (2020, background): “Recently, some works have already regarded GZSL as a missing data problem (Mishra et al. 2018; Verma et al. 2018; Xian et al. 2018b; Li et al. 2019).”
Generalized Zero-Shot Learning via Synthesized Examples (2017, methodology): “This also explicates why our model outperforms other recently proposed architectures, such as [3, 21] which dont have a feedback-driven mechanism in their generative models.”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, background): “Broadly, these models learn to generate/synthesize “artificial” examples from unseen classes [46, 10, 53, 29, 26, 41, 20], conditioning on their class attributes, and learn a classifier using these synthesized examples.”
A Zero-Shot Framework for Sketch-based Image Retrieval (2018, background): “This work has been submitted to a parallel conference and has been included in the Supplementary Material [28].”
Gradient Matching Generative Networks for Zero-Shot Learning (2019, background): “[45] proposes training a conditional Variational Auto-Encoder (cVAE), that learns to generate samples according to given class embeddings.”
Small Sample Learning in Big Data Era (2018, methodology): “Afterwards, Mishra et al. (2017) trained a conditional VAE (Sohn et al., 2015) to learn the underlying probability distribution of the image features conditioned on the class embedding vector.”
Bookworm continual learning: beyond zero-shot learning and continual learning (2020, methodology): “Recent works also use feature generators to synthesize features of unseen classes (Xian et al., 2018b; Mishra et al., 2018; Xian et al., 2019).”
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders (2018, methodology): “After individual VAEs learn to encode features of only their specific datatype for some epochs, we also start to compute cross- and distribution alignment losses. δ is increased from epoch 6 to epoch 22 by a rate of 0.54 per epoch, while γ is increased from epoch 21 to 75 by 0.044 per epoch.”
Deep Learning for Anomaly Detection: A Survey (2019, methodology): “However several approaches of using ZSL in anomaly and novelty detection are shown to produce state-of-the-art results (Mishra et al. [2017], Socher et al. [2013], Xian et al. [2017], Liu et al. [2017], Rivero et al. [2017]).”
Adaptive Cross-Modal Few-Shot Learning (2019, methodology): “Data synthetic methods rely on generative models to generate image or image feature as data augmentation [60, 57, 31, 54] for unseen data to train the mapping function for more robust alignment.”
Large-Scale Few-Shot Learning: Knowledge Transfer With Class Hierarchy (2019, methodology): “Furthermore, our feature learning model can be easily extended to the closely related zero-shot learning (ZSL) problem [32, 17, 29, 31, 14].”
Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks (2017, background): “Recently, several ZSL models adopt generative model for data augmentation of unseen classes [38, 6].”
Contrastive Embedding for Generalized Zero-Shot Learning (2021, methodology): “To further mitigate the data imbalance problem, feature generation methods learn to complement the visual samples for unseen classes [7, 50, 38, 70, 72, 56, 59, 64].”
Adaptive and Generative Zero-Shot Learning (2021, background): “Generative approaches (Chen et al., 2018; Felix et al., 2018; Kumar Verma et al., 2018; Mishra et al., 2018; Schonfeld et al., 2019; Paul et al., 2019; Xian et al., 2019) tackle the problem by synthesizing visual features of unseen classes from their semantic features with generative models like Gen”
A Review of Generalized Zero-Shot Learning Methods (2020, methodology): “Studies [165], [167] use CVAE-based frameworks to generate visual features for GZSL.”