Learning a Deep Embedding Model for Zero-Shot Learning

By Li Zhang, Tao A. Xiang, Shaogang Gong. 2016

764 citations recorded by the source corpus.

Zero-shot learning (ZSL) models rely on learning a joint embedding space where both textual/semantic description of object classes and visual representation of object images can be projected to for nearest neighbour search. Despite the success of deep neural networks that learn an end-to-end model between text and images in other vision problems such as image captioning, very few deep ZSL model exists and they show little advantage over ZSL models that utilise deep feature representations but do not learn an end-to-end embedding. In this paper we argue that the key to make deep ZSL models succeed is to choose the right embedding space. Instead of embedding into a semantic space or an intermediate space, we propose to use the visual space as the embedding space. This is because that in this space, the subsequent nearest neighbour search would suffer much less from the hubness problem and thus become more effective. This model design also provides a natural mechanism for multiple semantic modalities (e.g.,~attributes and sentence descriptions) to be fused and optimised jointly in an end-to-end manner. Extensive experiments on four benchmarks show that our model significantly outperforms the existing models.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning to Compare: Relation Network for Few-Shot Learning (2017, methodology): “This DNN is pre-trained on ILSVRC 2012 1K classification without fine-tuning, as in recent deep ZSL works [25, 30, 45].”
Rectification-based Knowledge Retention for Continual Learning (2021, background): “The average H values are 10.2%, 36.73%, 38.03%, 36.73% and 42.48% for base, SFT, L1, L2 and LZSL [30] respectively.”
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, background): “larly, [35] argues that the visual feature space is more discriminative than the semantic space, thus it proposes an end-to-end deep embedding model which maps semantic”
Feature Generating Networks for Zero-Shot Learning (2017, methodology): “We compare our method with [49] using the same evaluation protocol as our paper, i.”
A Survey of Zero-Shot Learning (2019, methodology): “Some methods use other projection spaces [134, 171], while some others are under the CTIT learning setting [29].”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, neutral): “22] 8.8 18.0 11.8 7.8 54.0 13.6 0.4 80.9 0.9 1.1 82.2 2.2 LATEM [51] 7.3 71.7 13.3 15.2 57.3 24.0 0.1 73.0 0.2 11.5 77.3 20.0 DEVISE [12] 13.4 68.7 22.4 23.8 53.0 32.8 4.9 76.9 9.2 17.1 74.7 27.8 DEM [59] 32.8 84.7 47.3 19.6 57.9 29.2 11.1 75.1 19.4 30.5 86.4 45.1 ZSKL [58] 18.3 79.3 29.8 21.6 52.8 ”
Attentive Region Embedding Network for Zero-Shot Learning (2019, methodology): “Other embedding based approaches include DEVICE [13], SJE [3], CMT [39], ESZSL [37], SAE [20], and DEM [52]; for a more detailed description of them, refer to [46].”
Leveraging the Invariant Side of Generative Zero-Shot Learning (2019, neutral): “iSE [7] 39.8 54.2 52.0 45.9 56.5 SJE [2] 32.9 65.6 53.9 53.4 53.7 ESZSL [29] 38.3 58.2 53.9 51.0 54.5 ALE [1] 39.7 59.9 54.9 48.5 58.1 SYNC [4] 23.9 54.0 55.6 - 56.3 SAE [13] 8.3 53.0 33.3 - 40.3 DEM [37] 35.0 68.4 51.7 - 61.9 GAZSL [40] 41.1 68.2 55.8 60.5 61.3 f-CLSWGAN [34] 40.5 68.2 57.3 67.2 60”
Gradient Matching Generative Networks for Zero-Shot Learning (2019, background): “Similarly, [37, 38, 39] learn a mapping from semantic embeddings to visual features, and, [40, 41, 42] learn a datadriven metric for comparing similarities between features and semantic embeddings.”
Lifelong Zero-Shot Learning (2020, background): “Zero-Shot Learning [Socher et al. , 2013; Zhang et al. , 2017; Zhao et al. , 2018; Chen et al. , 2018 ] has become a popular research topic, which aims to recognize unseen classes without any labeled training data.”
Zero-Shot Sketch-Image Hashing (2018, background): “Zero-shot learning [19, 31, 75, 76, 56, 46, 7, 2, 64, 3, 34, 11, 8, 25, 74, 14, 35, 65, 27, 68, 40] is also related to our work, though it does not originally focus on cross-modal retrieval.”
Locality and compositionality in zero-shot learning (2019, methodology): “Our setup is representative of the current state of ZSL models, most of which (Akata et al., 2015; Changpinyo et al., 2016; Kodirov et al., 2017; Zhang et al., 2017; Sung et al., 2018) rely on metric learning by applying two steps: (1) learning a suitable embedding function that maps data samples…”
A Simple Exponential Family Framework for Zero-Shot Learning (2017, background): “, mapping the class-attribute to the observed feature space [32,37].”
A Variational Autoencoder with Deep Embedding Model for Generalized Zero-Shot Learning (2020, background): “…on building an embedding model that first learns the mapping between visual feature space and semantic space (Akata et al. 2013; Frome et al. 2013; Lampert, Nickisch, and Harmeling 2014; Romera-Paredes and Torr 2015; Reed et al. 2016; Zhang, Xiang, and Gong 2017; Sung et al. 2018; Wei et al. 2019).”
Consistent Structural Relation Learning for Zero-Shot Segmentation (2020, background): “Zero-Shot Learning ZSL [20, 21] aims to recognize unseen classes with no training examples by leveraging the semantic label embeddings ( e.g ., word embeddings or attribute vectors) as side information.”
Region Graph Embedding Network for Zero-Shot Learning (2020, background): “DEM [66], CMT [41], SJE [4], and DEVICE [14] are also competitive embedding based models.”