Few-Shot Object Detection via Feature Reweighting
By Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, Trevor Darrell. 2018
862 citations recorded by the source corpus.
Conventional training of a deep CNN based object detector demands a large number of bounding box annotations, which may be unavailable for rare categories. In this work we develop a few-shot object detector that can learn to detect novel objects from only a few annotated examples. Our proposed model leverages fully labeled base classes and quickly adapts to novel classes, using a meta feature learner and a reweighting module within a one-stage detection architecture. The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples. The reweighting module transforms a few support examples from the novel classes to a global vector that indicates the importance or relevance of meta features for detecting the corresponding objects. These two modules, together with a detection prediction module, are trained end-to-end based on an episodic few-shot learning scheme and a carefully designed loss function. Through extensive experiments we demonstrate that our model outperforms well-established baselines by a large margin for few-shot object detection, on multiple datasets and settings. We also present analysis on various aspects of our proposed model, aiming to provide some inspiration for future few-shot detection works.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Trevor Darrell
- Dacheng Tao
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Jiashi Feng
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Bernt Schiele
- Jitendra Malik
- Andrew Zisserman
In Radiology, Nuclear Medicine and Imaging
- Jiashi Feng
- Weicheng Kuo
- Nick Barnes
- Shafin Rahman
- Martial Hebert
- Yu-Xiong Wang
- Tsung-Yi Lin
- Deva Ramanan
In Meta & Continual Learning
- Salman Hameed Khan
- Amos J. Storkey
- Yang Xiao
- Vincent Lepetit
- Renaud Marlet
- Lorenzo Pellegrini
- Antreas Antoniou
- Trevor Darrell
Citations in the corpus
- Grounded Language-Image Pre-training (2021, methodology): “We vary the amount of task-specific annotated data, from zero-shot (no data provided), to X-shot (providing at least X examples per category [22, 57, 61]), to using all data in the training set.”
- Learning Open-World Object Proposals without Learning to Classify (2021, background): “This line of research is also related to few-shot [10, 31, 67, 68, 71] and zero-shot detection [6, 20, 43, 70] methods, which attempt to detect novel classes given only a few samples or class descriptions.”
- Incremental Few-Shot Instance Segmentation (2021, methodology): “This approach has also been extended to FSOD (e.g., [13]) and FSIS (e.g., [8, 39]) by considering all objects in an image as queries and having a single support set per-image instead of per-query.”
- Incremental Object Detection via Meta-Learning (2020, neutral): “[32] learn to re-weight the RoI features and backbone features of Faster R-CNN [2] and YOLO [3], respectively.”
- Incremental Few-Shot Object Detection (2020, neutral): “The results on COCO in Table 1 show that, while not directly comparable due to using different detection backbones and/or data split, ONCE approaches the performance of the two state-of-the-art models [22, 53].”
- Meta-Learning in Neural Networks: A Survey (2020, background): “ncremental addition of new few-shot classes [128], [165]. Object Detection Building on the rapid progress in fewshot classification, recent work has also generalized to fewshot object detection [165], [194], often using feed-forward hypernetwork-based approaches to embed support set images and synthe”
- Frustratingly Simple Few-Shot Object Detection (2020, methodology): “We describe the existing meta-learning based few-shot object detection networks, including FSRW (Kang et al., 2019), Meta R-CNN (Yan et al., 2019) and MetaDet (Wang et al., 2019b), in this section to draw comparisons with our approach.”
- Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning (2019, methodology): “[21] follows fullimage meta-learning principle to address this problem.”
- Meta-Learning to Detect Rare Objects (2019, methodology): “In addition, we compare with a recent metalearning based few-shot detection approach that reweights the feature maps from a pre-trained base YOLOv2 detector [29].”
- 3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding (2022, background): “Research does not only cover the advancement of performance under classic learning paradigms (supervised, offline training) [20, 21, 22, 23, 24, 25, 26, 27], but also encompasses areas such as weakly-supervised learning [28, 29, 30, 31], few-shot learning [32, 33, 34, 35, 36, 37, 38], and semi-super”
- The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation (2020, background): “For example, AP 1 corresponds to the commonly referred few-shot object detection regime [8], [23], [25].”
- DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, methodology): “As one of the research branches, fewshot object detection (FSOD) is a much more challenging task than both few-shot classification and object detection [5, 19, 46, 51, 53].”
- Learning to Segment the Tail (2020, background): “Besides, the scale also raises major memory issues in the episodic training [41] adopted by recent meta-learning based methods [20, 29].”
- Context-Transformer: Tackling Object Confusion for Few-Shot Detection (2020, methodology): “Following the data setting in (Kang et al. 2019), we perform model pretraining on VOC07+12 training set (only 15 source-domain classes de-fined by (Kang et al. 2019)).”
- Multi-Scale Positive Sample Refinement for Few-Shot Object Detection (2020, methodology): “Following the settings in [16,41], object classes are divided into base classes with abundant data and novel classes with only a few training samples.”
- Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation (2021, background): “…samples and has gained significant improvement on the interpretation of natural images, including image classification [17], [18], object detection [19], [20], semantic segmentation [21], [22], etc. Inspired by their successful applications in natural images, extensive efforts had been made by…”
- Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector (2019, methodology): “In Table 5, we compare our approach 1 with Feature Reweighting [10] and Meta R-CNN [12] fsod means the model is fine-tuned on FSOD dataset.”