Synthesized Classifiers for Zero-Shot Learning

By Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha. 2016

755 citations recorded by the source corpus.

Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided. We propose to tackle this problem from the perspective of manifold learning. Our main idea is to align the semantic space that is derived from external information to the model space that concerns itself with recognizing visual features. To this end, we introduce a set of "phantom" object classes whose coordinates live in both the semantic space and the model space. Serving as bases in a dictionary, they can be optimized from labeled data such that the synthesized real object classifiers achieve optimal discriminative performance. We demonstrate superior accuracy of our approach over the state of the art on four benchmark datasets for zero-shot learning, including the full ImageNet Fall 2011 dataset with more than 20,000 unseen classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Learning Visual N-Grams from Web Data (2016, methodology): “, [7, 35, 69]), we simply apply the Flickr-trained models on a test set from a different dataset.”
Prototypical Networks for Few-shot Learning (2017, methodology): “The Synthesized Classifiers approach of [6] is a manifold learning technique that aligns the class meta-data space with the visual model space, and the method of Zhang and Saligrama [36] is a structured prediction approach trained on top of VGG-19 features [30].”
Learning to Compare: Relation Network for Few-Shot Learning (2017, neutral): “4 SYNC [6] 54.”
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, methodology): “5% ), SYNC [14] on CUB ( 55 .”
Zero-Shot Learning — The Good, the Bad and the Ugly (2017, methodology): “For sanity-check, we re-evaluate methods [21, 42, 39, 4, 31, 7] 2 using provided features and code.”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, background): “By aligning the visual and semantic definitions of objects, ZSL transfers the relationship between images and attributes learned from seen classes to unseen ones, so as to recognize a novel instance with only its category-wise attributes [7, 9].”
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild (2016, methodology): “[4], this setting has almost always been adopted for evaluating ZSL methods [10,11,12,13,14,15,8,16,17,18,19,20,21,22,23,24,25,26,27,28].”
Learning the Best Pooling Strategy for Visual Semantic Embedding (2020, methodology): “[10] used this approach for zero-shot image recognition [2, 24, 34], via matching visual embeddings with semantic word embeddings.”
Feature Generating Networks for Zero-Shot Learning (2017, background): “[50, 31, 8] learn unseen classes as a mixture of seen class proportions, and [2, 3, 14, 42, 45, 40, 16, 33, 1, 6, 17, 23] learn a compatibility between images and classes.”
Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions (2018, background): “Another line of approach has focused on learning generalizable instance embeddings [1, 4, 5, 13, 17, 26, 37, 40, 43] and uses those embeddings on simple classifiers such as nearest neighbor rules.”
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders (2018, methodology): “On the other hand, the classic ZSL methods DeViSE [6], SJE [2], ALE [1], EZSL [24] and LATEM [34] use a linear compatibility function or other similarity metrics to compare embedded visual and semantic features; CMT [27] and LATEM [34] utilize multiple neural networks to learn a non-linear embedding”
Locality and compositionality in zero-shot learning (2019, methodology): “Our setup is representative of the current state of ZSL models, most of which (Akata et al., 2015; Changpinyo et al., 2016; Kodirov et al., 2017; Zhang et al., 2017; Sung et al., 2018) rely on metric learning by applying two steps: (1) learning a suitable embedding function that maps data samples…”
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance (2018, background): “alone – termed zero-shot learning (ZSL) – has been the topic of increased interest within the community [17,16,10,21,29,36,31,2,11,3,25,5,14].”
Generalized Zero-Shot Learning with Deep Calibration Network (2018, methodology): “Comparison methods We choose to compare with many competitive or representative methods, including shallow methods DAP [32], ALE [1], SJE [3], ESZSL [42], ConSE [36], SynC [8], EXEM [9], ZSKL [57], and deep methods DeViSE [15], CMT [47], MTMDL [55], DEM [58].”
A Simple Exponential Family Framework for Zero-Shot Learning (2017, background): “Another popular approach for ZSL is based on modelling each unseen class as a linear/convex combination of seen classes [20] or of a set of “abstract” or “basis” classes [22,5].”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “Though research towards deep oneshot learning [94], [453] and deep zero-shot learning [454],”
Semantic Autoencoder for Zero-Shot Learning (2017, methodology): “Inspired by humans’ ZSL ability, there is a recent surge of interest in machine learning based ZSL for scaling up visual recognition to unseen object classes without the need for additional data collection [66, 48, 19, 49, 2, 13, 58, 1, 54, 22, 23, 40, 11].”
Generalising Fine-Grained Sketch-Based Image Retrieval (2019, background): “This motivates, for example, extensive research in zero-shot object recognition [11, 42, 6].”
Learning Embedding Adaptation for Few-Shot Learning (2018, background): “Another line of approach has focused on learning generalizable instance embeddings [1, 5, 6, 18, 28, 42, 44] and uses those embeddings on simple classifiers such as nearest neighbor classification rules.”