Zero-Shot Learning via Joint Latent Similarity Embedding

By Ziming Zhang, Venkatesh Saligrama. 2015

363 citations recorded by the source corpus.

Zero-shot recognition (ZSR) deals with the problem of predicting class labels for target domain instances based on source domain side information (e.g. attributes) of unseen classes. We formulate ZSR as a binary prediction problem. Our resulting classifier is class-independent. It takes an arbitrary pair of source and target domain instances as input and predicts whether or not they come from the same class, i.e. whether there is a match. We model the posterior probability of a match since it is a sufficient statistic and propose a latent probabilistic model in this context. We develop a joint discriminative learning framework based on dictionary learning to jointly learn the parameters of our model for both domains, which ultimately leads to our class-independent classifier. Many of the existing embedding methods can be viewed as special cases of our probabilistic model. On ZSR our method shows 4.90% improvement over the state-of-the-art in accuracy averaged across four benchmark datasets. We also adapt ZSR method for zero-shot retrieval and show 22.45% improvement accordingly in mean average precision (mAP).

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning to Compare: Relation Network for Few-Shot Learning (2017, neutral): “4 JLSE [47] FV A 80.”
Zero-Shot Object Detection (2018, background): “Zero-shot learning has been shown to be effective in situations where there is a lack of annotated data [12,14,31,38,53,54,59,60].”
Latent Embeddings for Zero-Shot Classification (2016, background): “c.5we present our experimental evaluation and in Sec.6we conclude. 2. Related Work We are interested in the problem of zero-shot learning where the test classes are disjoint from the training classes [14,20,21,30,42,43]. As visual information from such test classes is not available during training, ”
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild (2016, methodology): “Existing research on ZSL has been almost entirely focusing on this setting [4,10,11,12,13,14,15,8,16,17,18,19,20,21,22,23,24,25,26,27,28].”
Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs (2018, background): “The core idea is to represent each category with learned vector representations that can be mapped to visual classifiers [48, 44, 13, 41, 25, 15, 14, 18, 23, 4, 5, 55, 54].”
Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration (2019, background): “Sometimes, both the visual features and the semantic descriptors are mapped to a common feature space [9], [10].”
MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning (2018, background): “…et al. (2015)), SC_struct (Changpinyo et al. (2016)), LatEm (Xian et al. (2016)), LEESD (Ding et al. (2017)), SS-Voc (Fu and Sigal (2016)), DCL (Guo et al. (2017)), JLSE (Zhang and Saligrama (2016)), ESZSL (Romera-Paredes and Torr (2015)), RKT (Wang et al. (2016)) and MFMR-joint (Xu et al. (2017)).”
A Simple Exponential Family Framework for Zero-Shot Learning (2017, methodology): “Transductive Setting For transductive ZSL setting [9,35,36], we follow the procedure described in Section 2.”
Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection (2019, background): “Zero Shot Learning (ZSL) seeks to recognize novel visual categories that are unannotated in training data [21, 40, 22, 45].”
Hard-Aware Deeply Cascaded Embedding (2016, background): “Deep metric embedding has attracted increasing attention for various tasks , such as visual product search [6, 18, 23, 27, 35], face recognition [21, 32, 41, 4], local image descriptor learning [9, 2, 12, 24], person/vehicle reidentification [22, 7, 43, 17], zero-shot image classification [19, 45, 5”
Leveraging the Invariant Side of Generative Zero-Shot Learning (2019, background): “In addition, there are also some works to learn an intermediate space shared by the visual features and semantic features [4, 38, 39].”
TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning (2019, methodology): “Many [23, 52, 42, 55, 4, 12] have studied metric learning based objectives to jointly learn the task embed-dings and image embeddings, resulting in a similarity or compatibility score that can later be used for classiﬁcation [31, 42, 26, 1, 2, 12, 39].”
Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication (2016, methodology): “- 41.98 0.67 75.48 0:43 Lampert et al. [12] 38.16 57.23 - 72.00 Romera-Paredes et al. [2] 24.22 2:89 75.32 2:28 - 82.10 0:32 Zhang et al. [11] 46.23 0:53 76.33 0:83 30.41 0:20 82.50 1.32 Zhang et al. [47] 50.35 2:97 80.46 0:53 42.11 0:55 83.83 0.29 Ours w/o ML 47.25 0:48 73.81 0:13 33.87 0:98 74.91 ”
Small Sample Learning in Big Data Era (2018, background): “Zhang and Saligrama (2016) firstly learned an intermediate latent embedding based on dictionary learning to jointly learn the parameters of model for both domains that can not only accurately represent the observed data in each domain but also infer cross-domain statistical relationships when one…”
Learning a Deep Embedding Model for Zero-Shot Learning (2016, background): “6% higher than the best result reported so far [48].”
Decoupling Zero-Shot Semantic Segmentation (2021, methodology): “The key idea in zero-shot learning is to transfer kowledge from seen classes to unseen classes via semantic representation, such as the semantic attributes [2,24,28,30], concept ontology [16,37,39,47,48] and semantic word vectors [17, 41, 60].”
Attentive Region Embedding Network for Zero-Shot Learning (2019, methodology): “Most leading ZSL methods, whether end-to-end convolutional neural network (CNN) based [27, 42, 22], or deep feature-based approaches [19, 51, 26, 53, 54, 34], emphasize on learning the embedding between the (learned) global image (or feature) and the counterpart semantic vector.”