Semantic Autoencoder for Zero-Shot Learning

By Elyor Kodirov, Tao A. Xiang, Shaogang Gong. 2017

901 citations recorded by the source corpus.

Existing zero-shot learning (ZSL) models typically learn a projection function from a feature space to a semantic embedding space (e.g. attribute space). However, such a projection function is only concerned with predicting the training seen class semantic representation (e.g. attribute prediction) or classification. When applied to test data, which in the context of ZSL contains different (unseen) classes without training data, a ZSL model typically suffers from the project domain shift problem. In this work, we present a novel solution to ZSL based on learning a Semantic AutoEncoder (SAE). Taking the encoder-decoder paradigm, an encoder aims to project a visual feature vector into the semantic space as in the existing ZSL models. However, the decoder exerts an additional constraint, that is, the projection/code must be able to reconstruct the original visual feature. We show that with this additional reconstruction constraint, the learned projection function from the seen classes is able to generalise better to the new unseen classes. Importantly, the encoder and decoder are linear and symmetric which enable us to develop an extremely efficient learning algorithm. Extensive experiments on six benchmark datasets demonstrate that the proposed SAE outperforms significantly the existing ZSL models with the additional benefit of lower computational cost. Furthermore, when the SAE is applied to supervised clustering problem, it also beats the state-of-the-art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, methodology): “Second, we confirmedwith the authors of SAE [33] that they improved GoogleNet [75] by adding Batch Normalization and averaging 5 randomly cropped images to obtain better image features.”
Zero-Shot Object Detection (2018, methodology): “We also add an additional reconstruction loss to L , as suggested by Kodirov et al. [23], to regularize the semantic embeddings.”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, background): “[1, 12, 2, 38, 22] are based on computing a linear/bilinear compatibility function.”
Zero-Shot Semantic Segmentation (2019, background): “There are different variants in the literature on how the projections or the similarity measure are computed: simple linear projection [1, 2, 6, 16, 24, 36], non-linear multi-modal embeddings [39, 43] or even hybrid methods [8, 32, 42, 48].”
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval (2019, methodology): “We also compare our model with two SBIR methods, GN-Triplet [33] and DSH [21], and two zero-shot methods, SAE [18] and ZSH [41].”
Multi-modal Cycle-consistent Generalized Zero-Shot Learning (2018, methodology): “Examples of such approaches are ALE [16], DEVISE [17], SJE [18], ESZSL [19], and SAE [20].”
A Survey of Zero-Shot Learning (2019, methodology): “There are also methods [68, 76, 162] in which more than one space is adopted as the projection space.”
Leveraging the Invariant Side of Generative Zero-Shot Learning (2019, methodology): “Inspired by the human ability that one can recognize an object at the ﬁrst glance by only knowing some semantic descriptions of it, zero-shot learning [4, 6, 13, 17, 35] aims to learn a model with good generalization ability which can recognize unseen objects by only giving some semantic attributes.”
Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects (2019, background): “transductive ZSL [15, 20], multi-label ZSL [18, 31], zero-shot detection [29] etc.”
Uncertainty-Aware Learning for Zero-Shot Semantic Segmentation (2020, positive): “And the classic zero-shot image recognition techniques [1, 3, 6, 18, 29, 35, 47, 52, 54, 61, 65] are directly applied to learn from seen classes the visual-semantic mappings, which are then transferred to unseen ones.”
Small Sample Learning in Big Data Era (2018, methodology): “ selected the most condently predicted unlabeled instances to rene the ensemble network parameters, and Luo et al. (2018) rened the class prototypes instead of labels. Dierent with other methods, Kodirov et al. (2017) took the encoderdecoder paradigm, and they insist that it is very eective in mitig”
Contrastive Embedding for Generalized Zero-Shot Learning (2021, background): “These works typically learn to embed visual samples and the semantic descriptors to an embedding space [18, 1, 19, 2, 21, 35, 20, 58, 6, 36, 8] (e.”
Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection (2019, background): “Motivated by these concerns, recent works have proposed to synthesize unseen examples by means of generative models such as autoencoders [4, 19], GANs and adversarial methods [49, 20, 42, 16, 23, 36] that take semantic vectors as input, and output images.”
Domain-Invariant Projection Learning for Zero-Shot Recognition (2018, methodology): “(3) For the large-scale ImNet dataset, our model achieves a 4.5% improvement over the state-of-the-art SAE [24], showing its scalability to large-scale problems.”
Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration (2019, neutral): “4 SAE [14] 54.”
Zero-Shot Learning via Class-Conditioned Deep Generative Models (2017, methodology): “Somewhat similar to our VAE based approach, recently (Kodirov et al. 2017) proposed a semantic autoencoder for ZSL.”