Gradient Matching Generative Networks for Zero-Shot Learning

By Mert Bulent Sariyildiz, R. G. Cinbis. 2019

154 citations recorded by the source corpus.

Zero-shot learning (ZSL) is one of the most promising problems where substantial progress can potentially be achieved through unsupervised learning, due to distributional differences between supervised and zero-shot classes. For this reason, several works investigate the incorporation of discriminative domain adaptation techniques into ZSL, which, however, lead to modest improvements in ZSL accuracy. In contrast, we propose a generative model that can naturally learn from unsupervised examples, and synthesize training examples for unseen classes purely based on their class embeddings, and therefore, reduce the zero-shot learning problem into a supervised classification task. The proposed approach consists of two important components: (i) a conditional Generative Adversarial Network that learns to produce samples that mimic the characteristics of unsupervised data examples, and (ii) the Gradient Matching (GM) loss that measures the quality of the gradient signal obtained from the synthesized examples. Using our GM loss formulation, we enforce the generator to produce examples from which accurate classifiers can be trained. Experimental results on several ZSL benchmark datasets show that our approach leads to significant improvements over the state of the art in generalized zero-shot classification.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Learning Visual Representations with Caption Annotations (2020, background): “Pretraining CNNs on an external dataset has become standard practice in computer vision [8,22,50,52], especially for domains or tasks for which data is scarce.”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, methodology): “For the purpose of zero-shot image classification, GANs have also been used for synthesizing visual features of unseen classes from their semantic features [5, 36, 14, 44].”
Contrastive Embedding for Generalized Zero-Shot Learning (2021, methodology): “To further mitigate the data imbalance problem, feature generation methods learn to complement the visual samples for unseen classes [7, 50, 38, 70, 72, 56, 59, 64].”
Generalized Continual Zero-Shot Learning (2020, unknown): “[38, 41, 23, 37, 17, 46, 12, 30, 2, 31, 46, 37].”
Concept Generalization in Visual Representation Learning (2020, unknown): “Generalization across concepts is a crucial part of zero- [51, 55] and few-shot [61] learning.”
Dataset Condensation with Gradient Matching (2020, methodology): “Finally our method is related to the methods that recover images by projecting the feature activations back to the input pixel space [18], reconstruct the input image by matching the feature activations [19], recover private training images for given training gradients [20, 21], synthesize features ”
Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection (2019, background): “Motivated by these concerns, recent works have proposed to synthesize unseen examples by means of generative models such as autoencoders [4, 19], GANs and adversarial methods [49, 20, 42, 16, 23, 36] that take semantic vectors as input, and output images.”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, methodology): “For example, the method in [44] first generated features using word embeddings and random vectors, which was further improved by later works [7, 22, 28, 40, 45].”
Synthesized Feature based Few-Shot Class-Incremental Learning on a Mixture of Subspaces (2021, methodology): “Learning without forgetting using word vectors: Word vectors have shown promising success on various computer vision tasks such as zero-shot learning, few-shot learning, image/video captioning and visual question answering [15, 29, 39, 43, 9, 46, 47, 6, 5, 7].”