Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration

By Debasmit Das, C. S. George Lee. 2019

31 citations recorded by the source corpus.

Zero-shot learning (ZSL) for image classification focuses on recognizing novel categories that have no labeled data available for training. The learning is generally carried out with the help of mid-level semantic descriptors associated with each class. This semantic-descriptor space is generally shared by both seen and unseen categories. However, ZSL suffers from hubness, domain discrepancy and biased-ness towards seen classes. To tackle these problems, we propose a three-step approach to zero-shot learning. Firstly, a mapping is learned from the semantic-descriptor space to the image-feature space. This mapping learns to minimize both one-to-one and pairwise distances between semantic embeddings and the image features of the corresponding classes. Secondly, we propose test-time domain adaptation to adapt the semantic embedding of the unseen classes to the test data. This is achieved by finding correspondences between the semantic descriptors and the image features. Thirdly, we propose scaled calibration on the classification scores of the seen classes. This is necessary because the ZSL model is biased towards seen classes as the unseen classes are not used in the training. Finally, to validate the proposed three-step approach, we performed experiments on four benchmark datasets where the proposed method outperformed previous results. We also studied and analyzed the performance of each component of our proposed ZSL framework.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Subspace Regularizers for Few-Shot Class Incremental Learning (2021, background): “…information, a number of previous studies in vision and language have also investigated the effectiveness of directly learning a mapping from the space of semantic embeddings to the space of class weights (Das & Lee, 2019; Socher et al., 2013; Pourpanah et al., 2020; Romera-Paredes & Torr, 2015).”
A survey on face data augmentation for the training of deep neural networks (2019, methodology): “To compensate the insufﬁcient training data, domain adaptation transfers the knowledge learned from a source data distribution to a target data distribution [23–26].”
A Review of Generalized Zero-Shot Learning Methods (2020, unknown): “Similar to the calibrated stacking, scaled calibration [82] and probabilistic representation [83], [84] have been proposed to balance the trade-off between both seen and unseen classes.”
MetaSearch: Incremental Product Search via Deep Meta-Learning (2020, background): “Das and Lee [6] introduced a three-step approach to improve the performance of zero-shot learning for image classiﬁcation that outperformed the existing methods.”