Frustratingly Simple Few-Shot Object Detection
By Thomas E. Huang, Trevor Darrell, Joseph E. Gonzalez, Fisher Yu. 2020
747 citations recorded by the source corpus.
Detecting rare objects from a few examples is an emerging problem. Prior works show meta-learning is a promising approach. But, fine-tuning techniques have drawn scant attention. We find that fine-tuning only the last layer of existing detectors on rare classes is crucial to the few-shot object detection task. Such a simple approach outperforms the meta-learning methods by roughly 2~20 points on current benchmarks and sometimes even doubles the accuracy of the prior methods. However, the high variance in the few samples often leads to the unreliability of existing benchmarks. We revise the evaluation protocols by sampling multiple groups of training examples to obtain stable comparisons and build new benchmarks based on three datasets: PASCAL VOC, COCO and LVIS. Again, our fine-tuning approach establishes a new state of the art on the revised benchmarks. The code as well as the pretrained models are available at this https URL.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Zhe Gan
- Trevor Darrell
- Bernt Schiele
- Caiming Xiong
- Percy Liang
- Yoshua Bengio
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Bernt Schiele
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
In Meta & Continual Learning
- Bernt Schiele
- Yaoyao Liu
- K. J. Joseph
- Vineeth Nallure Balasubramanian
- Qianru Sun
- Salman Hameed Khan
- Han-Jia Ye
- De-chuan Zhan
Citations in the corpus
- ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models (2022, background): “In OD [67], linear probing typically means updating the linear heads for classification and localization (including centers) tasks only, while fine-tuning means update all model weights including the backbone and the detectors.”
- Grounded Language-Image Pre-training (2021, methodology): “[56] report the effectiveness of “linear probing” (i.”
- Learning Open-World Object Proposals without Learning to Classify (2021, methodology): “This line of research is also related to few-shot [33] and zeroshot detection[34], [35] methods, which attempt to detect novel classes given only a few samples or class descriptions.”
- Wanderlust: Online Continual Object Detection in the Real World (2021, background): “Nevertheless, challenges remain when detecting small objects [7], scaling to a large number of categories [15] or learning from only a few labeled examples [19, 44].”
- Incremental Few-Shot Instance Segmentation (2021, methodology): “Our non-incremental baseline approach extends the Two-Stage Fine-tuning (TFA, [36]) object-detection method introduced by Wan et al .”
- Towards Open World Object Detection (2021, background): “[62].”
- An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “There are different meta strategies, among which the gradient descent based methods are particularly promising for today’s neural networks [1, 13–15, 20, 25, 38, 70, 74, 81, 83, 84, 86].”
- Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “For a fair comparison, we consider the same 3 splits as in [23, 59, 64, 57], and for each run we only draw K random shots”
- GLIPv2: Unifying Localization and Vision-Language Understanding (2022, methodology): “Prompt tuning in a deep-fused model such as GLIPv2 is different from the conventional linear probing/prompt tuning setting [53, 44, 60] in shallow-interacting vision models such as CLIP.”
- The Art of Prompting: Event Detection based on Type Specific Prompts (2022, methodology): “…Detection: Two primary learning strategies in few-shot classification tasks are Meta-Learning (Kang et al., 2019; Li et al., 2021; Xiao and Marlet, 2020; Yan et al., 2019; Chowd-hury et al., 2021), and Metric Learning (Sun et al., 2021; Wang et al., 2020b; Zhang et al., 2021a; Agarwal et al., 2021).”
- 3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding (2022, background): “Research does not only cover the advancement of performance under classic learning paradigms (supervised, offline training) [20, 21, 22, 23, 24, 25, 26, 27], but also encompasses areas such as weakly-supervised learning [28, 29, 30, 31], few-shot learning [32, 33, 34, 35, 36, 37, 38], and semi-super”
- Distribution Alignment: A Unified Framework for Long-tail Visual Recognition (2021, methodology): “The similar idea is adopted in [38, 39, 23], which develop effective strategies for long-tail object detection tasks.”
- Visual Prompting via Image Inpainting (2022, methodology): “In this setting, the algorithm is trained on a labeled dataset of base classes, from which it should transfer to a set of novel classes given only a few training examples (like 10 or 30) [36, 27, 32, 53, 55, 48, 59, 2].”
- OW-DETR: Open-world Detection Transformer (2021, methodology): “When a set of new categories are introduced for the incremental learning stage at each episode in OWOD, motivated by the findings in [15, 28, 34], we employ an exemplar replay based finetuning to alleviate catastrophic forgetting of previously learned classes.”
- A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, neutral): “TFA [252] ICML’20 Resnet-101 Transfer learning 10AP 13.”
- Multi-Scale Positive Sample Refinement for Few-Shot Object Detection (2020, result): “esNet-101, respectively. It should be noted that due to better implementation and training strategy, our baseline achieves higher performance than SOTA, which is also confirmed by the very recent work [40]. PASCAL VOC. MPSR achieves 82.1%/82.7%/82.9% on base classes of three splits respectively befor”
- Online Structured Meta-learning (2020, background): “Despite the early success of meta-learning on various applications (e.g., computer vision [7, 40], natural language processing [14, 36]), almost all traditional meta-learning algorithms make the assumption that tasks are sampled from the same stationary distribution.”
- DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, methodology): “And TFA [46] proposes a simple approach based on transfer learning, that only fine-tunes the last layer of existing detectors on rare classes, which are comparable to the previous meta-based methods.”