Any-Shot Object Detection

By Shafin Rahman, Salman Hameed Khan, Nick Barnes, Fahad Shahbaz Khan. 2020

19 citations recorded by the source corpus.

Previous work on novel object detection considers zero or few-shot settings where none or few examples of each category are available for training. In real world scenarios, it is less practical to expect that 'all' the novel classes are either unseen or {have} few-examples. Here, we propose a more realistic setting termed 'Any-shot detection', where totally unseen and few-shot categories can simultaneously co-occur during inference. Any-shot detection offers unique challenges compared to conventional novel object detection such as, a high imbalance between unseen, few-shot and seen object classes, susceptibility to forget base-training while learning novel classes and distinguishing novel classes from the background. To address these challenges, we propose a unified any-shot detection model, that can concurrently learn to detect both zero-shot and few-shot object classes. Our core idea is to use class semantics as prototypes for object detection, a formulation that naturally minimizes knowledge forgetting and mitigates the class-imbalance in the label space. Besides, we propose a rebalanced loss function that emphasizes difficult few-shot cases but avoids overfitting on the novel classes to allow detection of totally unseen classes. Without bells and whistles, our framework can also be used solely for Zero-shot detection and Few-shot detection tasks. We report extensive experiments on Pascal VOC and MS-COCO datasets where our approach is shown to provide significant improvements.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning (2021, background): “Being inspired by the literature on zero-shot learning [7, 6, 5, 20, 37, 24, 25], given an image as input, we estimate the semantic word vectors for the input instead of directly predicting its class label.”
Zero-Shot Object Detection: Joint Recognition and Localization of Novel Concepts (2020, background): “Furthermore, the interesting extensions of ZSD setting, such as the any-shot detection [42], can lead to more practical scenarios close to the real-world.”
Synthesized Feature based Few-Shot Class-Incremental Learning on a Mixture of Subspaces (2021, methodology): “[25] has used semantic word vectors in the any-shot object detection problem in order to detect both unseen and few-shot objects simultaneously.”
DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, background): “Since previous detectors usually require a large amount of annotated data, few-shot detection has attracted more and more interest recently [2, 10, 12, 28, 31, 45, 47, 52, 54].”