Transductive Learning for Zero-Shot Object Detection

By Shafin Rahman, Salman Hameed Khan, Nick Barnes. 2019

84 citations recorded by the source corpus.

Zero-shot object detection (ZSD) is a relatively unexplored research problem as compared to the conventional zero-shot recognition task. ZSD aims to detect previously unseen objects during inference. Existing ZSD works suffer from two critical issues: (a) large domain-shift between the source (seen) and target (unseen) domains since the two distributions are highly mismatched. (b) the learned model is biased against unseen classes, therefore in generalized ZSD settings, where both seen and unseen objects co-occur during inference, the learned model tends to misclassify unseen to seen categories. This brings up an important question: How effectively can a transductive setting address the aforementioned problems? To the best of our knowledge, we are the first to propose a transductive zero-shot object detection approach that convincingly reduces the domain-shift and model-bias against unseen classes. Our approach is based on a self-learning mechanism that uses a novel hybrid pseudo-labeling technique. It progressively updates learned model parameters by associating unlabeled data samples to their corresponding classes. During this process, our technique makes sure that knowledge that was previously acquired on the source domain is not forgotten. We report significant 'relative' improvements of 34.9% and 77.1% in terms of mAP and recall rates over the previous best inductive models on MSCOCO dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning (2021, background): “Being inspired by the literature on zero-shot learning [7, 6, 5, 20, 37, 24, 25], given an image as input, we estimate the semantic word vectors for the input instead of directly predicting its class label.”
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation (2021, methodology): “Most ZSD methods align region features to pretrained text embeddings in base categories (Bansal et al., 2018; Demirel et al., 2018; Rahman et al., 2019; Hayat et al., 2020; Zheng et al., 2020).”
Open-Vocabulary Object Detection Using Captions (2020, background): “This has motivated recent papers to simplify the task by making unrealistic assumptions, such as the availability of test data during training [29], or the availability of unseen class annotations to filter images with unseen object instances [15].”
Zero-Shot Detection via Vision and Language Knowledge Distillation (2021, background): “[34] adopt a transductive setting to reduce the domain-shift against unseen classes.”
Simple Multi-dataset Detection (2021, methodology): “[28] used test-time training to progressively generate new class labels based on word embeddings.”
Improved Visual-Semantic Alignment for Zero-Shot Object Detection (2020, methodology): “A trans-ductive approach to leverage unlabelled target-domain data is proposed in (Rahman, Khan, and Barnes 2019).”
Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection (2019, methodology): “seen/unseen ratios in contrast to tabulating results for single splits with large seen/unseen ratios by [31, 1]); (3) consider multi-object image datasets, and results for other datasets such as F-MNIST that has clear black backgrounds as in [6] or single objects/image such as ImageNet as in [31, 30”
Any-Shot Object Detection (2020, methodology): “For the MSCOCO experiment, we adopt the 65/15 seen/unseen split setting used in [24, 25].”
A Review of Generalized Zero-Shot Learning Methods (2020, background): “alleviate the projection domain shift problem [35], [36], [37], [38].”
Zero-Shot Object Detection: Joint Recognition and Localization of Novel Concepts (2020, background): “Further, [40] proposed a transductive learning framework for ZSD.”