Generating Visual Representations for Zero-Shot Classification

By Max Bucher, Stéphane Herbin, Frédéric Jurie. 2017

158 citations recorded by the source corpus.

This paper addresses the task of learning an image classifier when some categories are defined by semantic descriptions only (e.g. visual attributes) while the others are defined by exemplar images as well. This task is often referred to as the Zero-Shot classification task (ZSC). Most of the previous methods rely on learning a common embedding space allowing to compare visual features of unknown categories with semantic descriptions. This paper argues that these approaches are limited as i) efficient discriminative classifiers can't be used ii) classification tasks with seen and unseen categories (Generalized Zero-Shot Classification or GZSC) can't be addressed efficiently. In contrast, this paper suggests to address ZSC and GZSC by i) learning a conditional generator using seen classes ii) generate artificial training examples for the categories without exemplars. ZSC is then turned into a standard supervised learning problem. Experiments with 4 generative models and 5 datasets experimentally validate the approach, giving state-of-the-art results on both ZSC and GZSC.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Delta-encoder: an effective sample synthesis method for few-shot object recognition (2018, methodology): “In [3] the authors suggested the usage of Denoising-Autoencoder (DAE) for zero-shot learning (Fig.”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, background): “This setting is popularly known as Zero-Shot Learning (ZSL), and has drawn a considerable interest recently [44, 32, 47, 5, 38, 53, 46, 27, 39, 7, 29, 4, 33, 10, 62, 41, 23].”
Domain-Invariant Projection Learning for Zero-Shot Recognition (2018, background): “(3) Generalized ZSL : The third ZSL setting that emerges recently [43, 7] is the generalized setting under which the test set contains data samples from both seen and unseen classes.”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, methodology): “For the purpose of zero-shot image classification, GANs have also been used for synthesizing visual features of unseen classes from their semantic features [5, 36, 14, 44].”
Insights from the Future for Continual Learning (2020, background): “Although several approaches exist for zero-shot learning, we will focus on generating a representation for the future classes [3, 22, 39].”
A Review of Generalized Zero-Shot Learning Methods (2020, unknown): “[112] used four different conditional GANs, i.”
A Survey of Zero-Shot Learning (2019, methodology): “The generative moment matching network (GMMN) [11], denoising autoencoder [11], adversarial autoencoder [11], conditional variational autoencoder-based architecture [141], and generative adversarial networks (GANs) [11, 156, 179] are examples.”
Zero-Shot Semantic Segmentation (2019, methodology): “These hyper-parameters are chosen with the “zero-shot cross-validation procedure” in [7].”
Multi-modal Cycle-consistent Generalized Zero-Shot Learning (2018, methodology): “[26] concerns the generation of realistic images, our proposed approach, similarly to [1, 8, 9], aims to generate visual representations, such as the features from a deep residual network [24] - the strategy based on visual representation has shown to produce more accurate GZSL classification result”
Small Sample Learning in Big Data Era (2018, background): “For example, Bucher et al. (2017) learned a conditional generator (e.g., conditional GAN (Odena et al., 2017), denoising auto-encoder (Bengio et al., 2013), and so on) for generating artificial training examples to address ZSL and GZSL problems.”
F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning (2019, methodology): “Recently, generating visual features of previously unseen classes [58, 5, 28, 11] has shown its potential to perform well on extremely imbalanced image collections.”
Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks (2017, background): “Recently, several ZSL models adopt generative model for data augmentation of unseen classes [38, 6].”
Gradient Matching Generative Networks for Zero-Shot Learning (2019, methodology): “Towards addressing this problem, several recent work have proposed generative models that can synthesize training samples for unseen classes and learn a classifier from real and/or synthesized examples [25, 26, 27, 28].”
Generalized Zero-Shot Learning via Synthesized Examples (2017, methodology): “Notably, since the classification model is trained using labeled examples from seen as well as unseen classes, at test time, it is not biased towards predicting seen classes in the GZSL setting, as also evidenced by recent work on leveraging synthesized unseen class examples [4, 8, 14, 15].”
Exemplar-Supported Generative Reproduction for Class Incremental Learning (2018, methodology): “Besides generating high-quality images [8, 10, 22, 24], GANs have been used in image classification [34], zeroshot learning [3, 35] etc.”