Zero-shot recognition with unreliable attributes

By Dinesh Jayaraman, Kristen Grauman. 2014

303 citations recorded by the source corpus.

In principle, zero-shot learning makes it possible to train a recognition model simply by specifying the category's attributes. For example, with classifiers for generic attributes like striped and four-legged, one can construct a classifier for the zebra category by enumerating which properties it possesses—even without providing zebra training images. In practice, however, the standard zero-shot paradigm suffers because attribute predictions in novel images are hard to get right. We propose a novel random forest approach to train zero-shot models that explicitly accounts for the unreliability of attribute predictions. By leveraging statistics about each attribute's error tendencies, our method obtains more robust discriminative models for the unseen classes. We further devise extensions to handle the few-shot scenario and unreliable attribute descriptions. On three datasets, we demonstrate the benefit for visual category learning with zero or few training examples, a critical domain for rare categories or categories defined on the fly.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Unified Contrastive Learning in Image-Text-Label Space (2022, methodology): “In this sense, our work is also related to conventional zero-shot classification [9, 28, 45, 62, 65, 66].”
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, background): “Early works of zero-shot learning [1], [15], [24], [25], [26] make use of the attributes within a two-stage approach to infer the label of an image that belong to one of the unseen”
An embarrassingly simple approach to zero-shot learning (2015, methodology): “In these experiments we compare 4 methods: DAP [21], ZSRwUA [16], ESZSL (Sect.”
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation (2021, background): “Early works (Farhadi et al., 2009; Rohrbach et al., 2011; Jayaraman & Grauman, 2014) use visual attributes to create a binary code-book representing categories, which is used to transfer learned knowledge to unseen categories.”
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild (2016, methodology): “Existing research on ZSL has been almost entirely focusing on this setting [4,10,11,12,13,14,15,8,16,17,18,19,20,21,22,23,24,25,26,27,28].”
Region Graph Embedding Network for Zero-Shot Learning (2020, background): “Early works [24, 18] on ZSL rely on learning attribute classiﬁers, based on which the class posterior of a test image is deduced.”
K-LITE: Learning Transferable Visual Models with External Knowledge (2022, background): “…to associate observed and non-observed classes through some form of auxiliary information, which can be either implicit such as pre-trained semantic embeddings [92, 78, 10], or explicit such as attributes [21, 43, 35], text [18, 19, 72, 70], knowledge graphs [91, 74], or rules and ontologies [23].”
Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs (2018, background): “Early work on zero-shot learning used attributes [11, 24, 19] to represent categories as vector indicating presence/absence of attributes.”
Normalization Matters in Zero-Shot Learning (2020, methodology): “[30] Dinesh Jayaraman and Kristen Grauman.”
Unsupervised Domain Adaptation for Zero-Shot Learning (2015, methodology): “Inspired by humans’ ability to recognise a new object category (class) without ever seeing a visual instance, zero-shot learning (ZSL) has received increasing interests [18, 9, 19, 23, 15, 1, 36, 11, 2].”
Zero-Shot Object Detection: Joint Recognition and Localization of Novel Concepts (2020, background): “This approach does not work consistently if the semantic vectors are noisy [18].”
Classifier and Exemplar Synthesis for Zero-Shot Learning (2018, background): “This is much more efficient than other ZSL algorithms that learn at the level of each individual training instance [16,37,51,2,83,17, 67,50,27,44,3,61,85,86,42].”
Zero-Shot Learning with Generative Latent Prototype Model (2017, unknown): “They are SJE [31], HAP [38], ZSLwUA [39], PST [13], TMV [21], AMP [26], UDA [40] and UDICA [16].”
Generating Visual Representations for Zero-Shot Classification (2017, methodology): “For SUN we experiment two different settings: one with 10 unseen classes as in [20], a second, more competitive, with ten different folds randomly chosen and averaged, as proposed by [8] (72/71 splits).”
A Simple Exponential Family Framework for Zero-Shot Learning (2017, methodology): “We use the same train/test split for this data set as used in [10].”
Semi-supervised Vocabulary-Informed Learning (2016, unknown): “[22] A low-level 48.”
Webly Supervised Learning Meets Zero-shot Learning: A Hybrid Approach for Fine-Grained Classification (2018, methodology): “Following [16], we use the standard train-test split with 707 fully-supervised (resp.”
Semantic Autoencoder for Zero-Shot Learning (2017, methodology): “Only SUN dataset has multiple splits.”
Zero-Shot Learning via Semantic Similarity Embedding (2015, methodology): “[14] investigated the attribute unreliability issue and proposed a random forest approach.”