Leveraging the Invariant Side of Generative Zero-Shot Learning

By Jingjing Li, Mengmeng Jing, Ke Lu, Zhengming Ding, Lei Zhu, Zi-Liang Huang. Computer Vision and Pattern Recognition, 2019

338 citations recorded by the source corpus.

Conventional zero-shot learning (ZSL) methods generally learn an embedding, e.g., visual-semantic mapping, to handle the unseen visual samples via an indirect manner. In this paper, we take the advantage of generative adversarial networks (GANs) and propose a novel method, named leveraging invariant side GAN (LisGAN), which can directly generate the unseen features from random noises which are conditioned by the semantic descriptions. Specifically, we train a conditional Wasserstein GANs in which the generator synthesizes fake unseen features from noises and the discriminator distinguishes the fake from real via a minimax game. Considering that one semantic description can correspond to various synthesized visual samples, and the semantic description, figuratively, is the soul of the generated features, we introduce soul samples as the invariant side of generative zero-shot learning in this paper. A soul sample is the meta-representation of one class. It visualizes the most semantically-meaningful aspects of each sample in the same category. We regularize that each generated sample (the varying side of generative ZSL) should be close to at least one soul sample (the invariant side) which has the same class label with it. At the zero-shot recognition stage, we propose to use two classifiers, which are deployed in a cascade way, to achieve a coarse-to-fine result. Experiments on five popular benchmarks verify that our proposed approach can outperform state-of-the-art methods with significant improvements.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Infectious Diseases

View Paper: https://www.semanticscholar.org/paper/64bd7b8297495af6971cfbb92e17f06d01bab146

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Infectious Diseases

Citations in the corpus

Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection (2019, background): “Motivated by these concerns, recent works have proposed to synthesize unseen examples by means of generative models such as autoencoders [4, 19], GANs and adversarial methods [49, 20, 42, 16, 23, 36] that take semantic vectors as input, and output images.”
Contrastive Embedding for Generalized Zero-Shot Learning (2021, neutral): “[4] 26.5 74.0 39.0 LisGAN [3] 34.3 68.2 45.7 IZF [5] 42.3 60.5 49.8 Our CE-GZSL 35.5 65.4 46.0”
Attribute Prototype Network for Zero-Shot Learning (2020, background): “For instance, APN + ABP outperforms LisGAN by 14 .”
A Review of Generalized Zero-Shot Learning Methods (2020, background): “Since then, the conditional GAN (CGAN) has been combined with different strategies to generative discriminative visual features for the unseen classes, including optimal transport -based approach [141], TFGNSCS [142] which is an extended version of f-CLSWGAN that considers transfer information, meta”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, methodology): “For example, the method in [44] first generated features using word embeddings and random vectors, which was further improved by later works [7, 22, 28, 40, 45].”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, methodology): “Broadly, these models learn to generate/synthesize “artificial” examples from unseen classes [46, 10, 53, 29, 26, 41, 20], conditioning on their class attributes, and learn a classifier using these synthesized examples.”
A Variational Autoencoder with Deep Embedding Model for Generalized Zero-Shot Learning (2020, background): “Recently, some works have already regarded GZSL as a missing data problem (Mishra et al. 2018; Verma et al. 2018; Xian et al. 2018b; Li et al. 2019).”