Feature Generating Networks for Zero-Shot Learning

By Yongqin Xian, Tobias Lorenz, Bernt Schiele, Zeynep Akata. 2017

1,041 citations recorded by the source corpus.

Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task. To circumvent the need for labeled examples of unseen classes, we propose a novel generative adversarial network (GAN) that synthesizes CNN features conditioned on class-level semantic information, offering a shortcut directly from a semantic descriptor of a class to a class-conditional feature distribution. Our proposed approach, pairing a Wasserstein GAN with a classification loss, is able to generate sufficiently discriminative CNN features to train softmax classifiers or any multimodal embedding method. Our experimental results demonstrate a significant boost in accuracy over the state of the art on five challenging datasets - CUB, FLO, SUN, AWA and ImageNet - in both the zero-shot learning and generalized zero-shot learning settings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Rectification-based Knowledge Retention for Continual Learning (2021, methodology): “The authors in [32] propose fCLSWGAN, which uses conditional Wasserstein GANs, to generate features for unseen classes.”
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “Here, we did not apply DKL, but rather we used the pre-trained features published by (Xian et al., 2018).”
Generative Feature Replay For Class-Incremental Learning (2020, methodology): “In [51], they propose a GAN architecture with a classifier on top of the generator, in order to generate features that are better suited for classification.”
Memory-Efficient Incremental Learning Through Feature Adaptation (2020, background): “Earlier work on feature generation, rather than image generation, focuses on zero-shot learning [3, 42].”
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders (2018, methodology): “Although the features generated by [36] improved GZSL significantly, GAN-based loss functions suffer from instability in training.”
Generalized Zero-Shot Learning via Aligned Variational Autoencoders (2019, methodology): “An orthogonal approach to GZSL is to augment data by generating artificial image features, such as [23] who proposed to generate image features via a conditional WGAN.”
Task-Driven Modular Networks for Zero-Shot Compositional Learning (2019, methodology): “Compositional zero-shot learning (CZSL) is a special case of zero-shot learning (ZSL) [23, 14, 13, 35].”
Conterfactual Generative Zero-Shot Semantic Segmentation (2021, background): “Previous works [6, 7, 8] have already revealed some rational ways to use this knowledge and build up a zero-shot model.”
Lifelong Zero-Shot Learning (2020, methodology): “For instance, f-CLSWGAN [Xian et al. , 2018b] was proposed by employing conditional Wasserstein GANs, which generated discriminative unseen visual features.”
Few-Shot Learning via Saliency-Guided Hallucination of Samples (2019, methodology): “Recent methods such as Feature Generating Networks [41] and Model Selection Network [43] hallucinate the training data for unseen classes via Generative Adversarial Networks (GAN).”
F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning (2019, methodology): “Among those, [5] uses GMMN [33], [63, 58] use GANs[17] and [28] em-”
Region Graph Embedding Network for Zero-Shot Learning (2020, background): “Moreover, thanks to the success of generative models [16, 71], several feature hallucinating approaches [49, 43, 63, 10] have been proposed for converting ZSL into conventional recognition tasks.”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, methodology): “Inspired by feature generation methods for zero-shot classification [7, 44], ZS3Net learns to generate pixel-wise features from semantic word embeddings.”
Adaptive Cross-Modal Few-Shot Learning (2019, methodology): “Data synthetic methods rely on generative models to generate image or image feature as data augmentation [60, 57, 31, 54] for unseen data to train the mapping function for more robust alignment.”
Adaptive and Generative Zero-Shot Learning (2021, background): “In addition, each of all the classes under consideration is characterized by semantic features such as attributes (Xian et al., 2018b) or text descriptions (Zhu et al., 2018) to specify and relate seen and unseen classes.”
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-Based Image Retrieval (2019, background): “Although, most of the deep neural network models in this domain are trained using a discriminative loss function, a few generative models also exist [52, 55, 8] that are used as a data augmentation mechanism.”
Leveraging the Invariant Side of Generative Zero-Shot Learning (2019, methodology): “For the real CNN features, we follow previous work [34] to extract 2048-dimensional features from ResNet-101 [10] which is pre-trained on ImageNet.”