An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

By Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha. 2016

619 citations recorded by the source corpus.

We investigate the problem of generalized zero-shot learning (GZSL). GZSL relaxes the unrealistic assumption in conventional zero-shot learning (ZSL) that test data belong only to unseen novel classes. In GZSL, test data might also come from seen classes and the labeling space is the union of both types of classes. We show empirically that a straightforward application of classifiers provided by existing ZSL approaches does not perform well in the setting of GZSL. Motivated by this, we propose a surprisingly simple but effective method to adapt ZSL approaches for GZSL. The main idea is to introduce a calibration factor to calibrate the classifiers for both seen and unseen classes so as to balance two conflicting forces: recognizing data from seen classes and those from unseen ones. We develop a new performance metric called the Area Under Seen-Unseen accuracy Curve to characterize this trade-off. We demonstrate the utility of this metric by analyzing existing ZSL approaches applied to the generalized setting. Extensive empirical studies reveal strengths and weaknesses of those approaches on three well-studied benchmark datasets, including the large-scale ImageNet with more than 20,000 unseen categories. We complement our comparative studies in learning methods by further establishing an upper bound on the performance limit of GZSL. In particular, our idea is to use class-representative visual features as the idealized semantic embeddings. We show that there is a large gap between the performance of existing approaches and the performance limit, suggesting that improving the quality of class semantic embeddings is vital to improving ZSL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Experience Grounds Language (2020, background): “These representations allow for pixel level masks and skeletal modeling, and can be extended to zero-shot settings targeting all 20,000 ImageNet categories (Chao et al., 2016; Changpinyo et al., 2017).”
Low-Shot Learning from Imaginary Data (2018, neutral): “[3].”
Learning to Compare: Relation Network for Few-Shot Learning (2017, background): “‘generalised zero-shot learning’ setting which considers testing data as containing samples from both seen and unseen classes has therefore begun to gain increasing interest. We follow the setting of [6]. Speciﬁcally, we hold out 20% of the samples from the seen classes and mix them with the data sa”
Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning (2021, methodology): “To evaluate the contribution of the base and novel instances in the ﬁnal accuracy, we also report the Harmonic Mean (HM) [3] of the accuracy of the base and novel classes.”
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, background): “Although zero-shot versus generalized zero-shot learning evaluation works exist [3], [63] in the literature, our work stands out in multiple aspects.”
Zero-Shot Learning — The Good, the Bad and the Ugly (2017, methodology): “[8] provides a comparison between five methods evaluated on three datasets including ImageNet with three standard splits and proposes a metric to evaluate generalized zero-shot learning performance.”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, methodology): “Chao et al. (2016) propose a calibration-agnostic criterion for generalized zero-shot learning.”
Zero-Shot Object Detection (2018, neutral): “ative results to validate the methods proposed. Some of the limitations of the presented work, and areas for future work, are as follows. We did not address the case of generalized zero-shot learning [63] where testing is done in the presence of seen classes as well. In such cases some form of norma”
Conditional Prompt Learning for Vision-Language Models (2022, background): “, to recognize novel classes by training only on base classes [3,51,54,57].”
Recent Advances in Open Set Recognition: A Survey (2018, background): “Therefore, some researchers have begun to pay attention to the more generalized ZSL (G-ZSL) [21], where the testing samples come from both known known and unknown known classes.”
Feature Generating Networks for Zero-Shot Learning (2017, background): “In this work, we are focusing on arguably the most extreme case of lacking data, namely zero-shot learning [24, 46, 9], where the task is to learn to classify when no labeled examples of certain classes are available during training.”
Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions (2018, methodology): “To calibrate the prediction score on SEEN and UNSEEN classes [6, 46], we select a constant seen/unseen class probability over the validation set, and subtract this calibration factor from seen classes’ prediction score.”
Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings (2019, neutral): “Accordingly, all previous results can be seen as a Generalised Zero-Shot Learning (GZSL) [4] set-up: there exists both actions in the test sets that have been seen in training and actions that have not.”
Exploring the Challenges towards Lifelong Fact Learning (2018, background): “th Semantic and Random splits are provided) across different tasks. We named that evaluation Generalized lifelong learning (G-LLL), in line with the idea of Generalized zero-shot learning proposed in [3]. We detail the evaluation metric in Sec. 5.1. Advantages of a Visual-Semantic Embedding. As illu”
A Meta-Learning Framework for Generalized Zero-Shot Learning (2019, background): “The generalized ZSL (GZSL) [46, 6, 52, 53] problem is arguably a very realistic and challenging problem wherein, unlike the ZSL problem, the training (seen) and the test (unseen) classes are not disjoint.”
Classifier and Exemplar Synthesis for Zero-Shot Learning (2018, positive): “Table 5 Comparison between existing ZSL approaches on the task of GZSL based on the Area Under Seen-Unseen accuracy Curve (AUSUC) [10] on small datasets.”
Learning and the Unknown: Surveying Steps toward Open World Recognition (2019, background): “The work in this area has received hundreds of citations from a wide range of application areas that need, or are using, OSR including: • Audio analysis (Battaglino, Lepauloux, and (Chiachia et al. 2014; Pinto et al. 2015; Rattani, Scheirer, and Ross 2015; Juefei-Xu and Savvides 2016; G¨unther et al”
Attentive Region Embedding Network for Zero-Shot Learning (2019, background): “ii) Our initial results typically achieve a high tr, but a low ts, which indicates that calibrated stacking [6] is needed.”