Attribute-Based Detection of Unfamiliar Classes with Humans in the Loop

By Catherine Wah, Serge J. Belongie. 2013

30 citations recorded by the source corpus.

Recent work in computer vision has addressed zero-shot learning or unseen class detection, which involves categorizing objects without observing any training examples. However, these problems assume that attributes or defining characteristics of these unobserved classes are known, leveraging this information at test time to detect an unseen class. We address the more realistic problem of detecting categories that do not appear in the dataset in any form. We denote such a category as an unfamiliar class, it is neither observed at train time, nor do we possess any knowledge regarding its relationships to attributes. This problem is one that has received limited attention within the computer vision community. In this work, we propose a novel approach to the unfamiliar class detection task that builds on attribute-based classification methods, and we empirically demonstrate how classification accuracy is impacted by attribute noise and dataset "difficulty," as quantified by the separation of classes in the attribute space. We also present a method for incorporating human users to overcome deficiencies in attribute detection. We demonstrate results superior to existing methods on the challenging CUB-200-2011 dataset.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, background): “Considerable work has also gone into the challenging task of curating finegrained datasets [4,58,27,30,31,59,65,60,70] and developing interactive methods for recognition with a human in the loop [6,62,61,63].”
Label-Embedding for Image Classification (2015, neutral): “Attributes have been used for image description [19], [18], [9], caption generation [27], [41], face recognition [29], [51], [10], image retrieval [28], [56], [15], action recognition [32], [69], novelty detection [62] and object classiﬁcation [30], [18], [64], [65], [34], [54], [38].”
Crowdsourcing in Computer Vision (2016, unknown): “Wah et al. [2014] also ask the user to compare similarity, by marking which of a set of images is most similar to the query image. Wah et al. [2015] request similarity comparisons on localized image patches of bird species. They first select discriminative regions, then model the probability that pa”
Local Novelty Detection in Multi-class Recognition Problems (2015, methodology): “As done in [36], we extract features from patches around ground truth part locations but restrict feature representations as in [13] to histograms over both quantized OpponentSIFT descriptors [31] and color name descriptors [38].”
Multi-class Open Set Recognition Using Probability of Inclusion (2014, background): “One-class models are typically less effective than binary classifiers [5,45,54].”