Transductive Multi-View Zero-Shot Learning

By Yanwei Fu, Timothy M. Hospedales, Tao A. Xiang, Shaogang Gong. 2015

482 citations recorded by the source corpus.

Most existing zero-shot learning approaches exploit transfer learning via an intermediate semantic representation shared between an annotated auxiliary dataset and a target dataset with different classes and no annotation. A projection from a low-level feature space to the semantic representation space is learned from the auxiliary dataset and applied without adaptation to the target dataset. In this paper we identify two inherent limitations with these approaches. First, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/ domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. The second limitation is the prototype sparsity problem which refers to the fact that for each target class, only a single prototype is available for zero-shot learning given a semantic representation. To overcome this problem, a novel heterogeneous multi-view hypergraph label propagation method is formulated for zero-shot learning in the transductive embedding space. It effectively exploits the complementary information offered by different semantic representations and takes advantage of the manifold structures of multiple representation spaces in a coherent manner. We demonstrate through extensive experiments that the proposed approach (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) significantly outperforms existing methods for both zero-shot and N-shot recognition on three image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.

Subfields: Artificial Intelligence, Computational Mechanics, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Meta & Continual Learning

Citations in the corpus

Meta-Learning for Semi-Supervised Few-Shot Classification (2018, background): “Our algorithm is also related to transductive learning (Vapnik, 1998; Joachims, 1999; Fu et al., 2015), where the base classifier gets refined by seeing the unlabeled examples.”
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, background): “Apart from the inductive zero-shot learning set-up where the model has no access to neither visual nor side-information of unseen classes, transductive zero-shot learning approaches [28], [44], [45], [46], [47], [48], [49] use visual or semantic information of both seen and unseen classes without ha”
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild (2016, methodology): “Existing research on ZSL has been almost entirely focusing on this setting [4,10,11,12,13,14,15,8,16,17,18,19,20,21,22,23,24,25,26,27,28].”
Feature Generating Networks for Zero-Shot Learning (2017, background): “On the other hand, instead of using only labeled data, [15, 38, 26] leverage unlabeled data from unseen classes in the transductive setting.”
Semi-supervised Vocabulary-Informed Learning (2016, methodology): “ZSL was first explored with use of attributebased semantic representations [11, 15, 17, 18, 24, 32].”
Semantic Autoencoder for Zero-Shot Learning (2017, background): “Existing ZSL models mostly suffer from the projection domain shift problem [21].”
Classifier and Exemplar Synthesis for Zero-Shot Learning (2018, background): “heir choices of common spaces or compatibility functions. The linear or bilinear compatibility functions are extensively used [2,3,17, 61,40]. Some propose to use canonical correlation analysis (CCA) [18,19,42]. Nonlinear methods are scarce but have also been explored such as dictionary learning and”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, methodology): “Many ZSL methods [1, 8, 9, 11, 27, 32, 33, 37, 46] attempted to learn a mapping between feature space and semantic embedding space.”
A Review of Generalized Zero-Shot Learning Methods (2020, methodology): “space using data samples from the seen classes without any adaptation to the unseen classes causes the projection domain shift problem [42], [63], [66].”
Generalized Zero-Shot Learning with Deep Calibration Network (2018, background): “Another related setting is transductive zero-shot learning [18, 49] which exploits the unlabeled target data at training.”
Learning Robust Visual-Semantic Embeddings (2017, methodology): “Expand the test-time search space: Note that most of the methods [9, 41, 29, 49, 2, 50, 48, 7, 10] consider that, at test time, queries come from only test classes.”
Deep Learning for Free-Hand Sketch: A Survey (2020, methodology): “Similar to natural photo zero-shot learning/recognition [256], [257], [258], ZS-SBIR systems aim to enable query and retrieval of categories that are from unseen categories.”
Zero-Shot Learning via Class-Conditioned Deep Generative Models (2017, methodology): “Other methods that can leverage unlabeled data include (Fu et al. 2015a; Rohrbach et al. 2013; Li et al. 2015; Zhao et al. 2016).”
A Simple Exponential Family Framework for Zero-Shot Learning (2017, background): “In another related work, [8] leverage unlabeled data in a transductive ZSL framework to handle the domain shift problem.”
MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning (2018, methodology): “The label embedding of each class is a one-hot vector in FSL, while we use auxiliary knowledge Lampert et al. (2013); Fu et al. (2015b) to embed each label to be a semantic vector in ZSL.”
Region Graph Embedding Network for Zero-Shot Learning (2020, background): “Additionally, issues with domain bias [15, 8] still exist, meaning that the learned models merely rely on the seen categories, while ignoring the available unseen attributes.”
A Generative Model for Zero Shot Learning Using Conditional Variational Autoencoders (2017, background): “The challenges faced in learning such a mapping are well documented, the primary issue being that of domain shift first identified by [12].”
Attentive Region Embedding Network for Zero-Shot Learning (2019, background): “An attentive region embedding network (AREN) is proposed for solving the challenging ZSL/GZSL task, which consists of two branches, i.e., the upper stream attentive region embedding (ARE) and the bottom stream (attention guided) compressed second-order embedding (ACSE).”