Incremental Few-Shot Object Detection

By Juan-Manuel Pérez-Rúa, Xiatian Zhu, Timothy M. Hospedales, Tao A. Xiang. 2020

263 citations recorded by the source corpus.

Existing object detection methods typically rely on the availability of abundant labelled training samples per class and offline model training in a batch mode. These requirements substantially limit their scalability to open-ended accommodation of novel classes with limited labelled training data, both in terms of model accuracy and training efficiency during deployment. We present the first study aiming to go beyond these limitations by considering the Incremental Few-Shot Detection (iFSD) problem setting, where new classes must be registered incrementally (without revisiting base classes) and with few examples. To this end we propose OpeN-ended Centre nEt (ONCE), a detector designed for incrementally learning to detect novel class objects with few examples. This is achieved by an elegant adaptation of the efficient CentreNet detector to the few-shot learning scenario, and meta-learning a class-wise code generator model for registering novel classes. ONCE fully respects the incremental learning paradigm, with novel class registration requiring only a single forward pass of few-shot training samples, and no access to base classes – thus making it suitable for deployment on embedded devices, etc. Extensive experiments conducted on both the standard object detection (COCO, PASCAL VOC) and fashion landmark detection (DeepFashion2) tasks show the feasibility of iFSD for the first time, opening an interesting and very important line of research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Ziwei Liu
4. Tao A. Xiang
5. Wei-Lun Chao
6. Bernt Schiele
7. Da-Wei Zhou
8. Zeynep Akata

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Bernt Schiele
4. Trevor Darrell
5. Yi Yang
6. Xiaogang Wang
7. Ming-Hsuan Yang
8. Yu Qiao

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Liyuan Wang
4. Xingxing Zhang
5. Jun Zhu
6. Hang Su
7. Da-Wei Zhou
8. Wei-Lun Chao

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Besides, there are many other related settings, such as incremental few-shot detection [332], where a pre-trained detection model registers new classes with only a few annotated data; and open world object detection [193], where the model needs to identify potential objects of unknown classes and re”
• Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation (2022, background): “[18] studied object detection based on class-incremental learning on Faster RCNN detector with emphasis on few-shot scenarios, which is also the focus of ONCE algorithm [29].”
• Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object Detection (2021, neutral): “Recently, several incremental learning methods [40, 27, 4, 10, 20, 31] are proposed to overcome catastrophic forgetting in the object detection task.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “Aside from knowledge distillation based continual learning with sufficient data instances per each new task, several recent works [72, 73, 74] have focused on few-shot continual learning where only a few data instances are given for each new task.”
• Wanderlust: Online Continual Object Detection in the Real World (2021, methodology): “In the object detection domain, several incremental object detection algorithms [23, 27, 32, 41] adopt existing object detection datasets such as PASCAL VOC [11] and MS COCO [26] for evaluation.”
• Incremental Few-Shot Instance Segmentation (2021, methodology): “Additionally, we compare the object detection performance to the only known incremental FSOD method, ONCE [23].”
• SID: Incremental Learning for Anchor-Free Object Detection via Selective and Inter-Related Distillation (2020, methodology): “Perez-Rua et al. (2020) designed a few-shot incremental detector based on CenterNet, but their method mainly focus on few-shot and meta learning.”
• MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “Incremental learning is usually conducted under the task-incremental [72], [73] or the class-incre-mental learning scenarios [2], [70], [74], [75], [76], [77].”
• Meta-Learning in Neural Networks: A Survey (2020, methodology): “Building on progress in few-shot classification, few-shot object detection [175], [205] has been demonstrated, often using feed-forward hypernetworkbased approaches to embed support set images and synthesize final layer classification weights in the base model.”
• Few-Shot Learning With a Strong Teacher (2021, methodology): “Meta-learning based few-shot learning has been applied in various fields, such as unsupervised learning [66], human motion prediction [20], domain adaptation [67], image generation [68], and object detection [69], [70].”
• Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery (2021, methodology): “Inspired by the effectiveness of regularization methods, some pioneer works have introduced the distillation strategy into other fields, such as object detection [48], [49] and semantic segmentation [32]–[35].”
• A Few Guidelines for Incremental Few-Shot Segmentation (2020, methodology): “few examples of new classes are provided in absence of old data for two reasons: (i) training a network on few samples strongly leads to overfitting [13], as it is not able to generalize to novel instances, and (ii) fine-tuning the model on new data leads to a strong degradation in performance for t”
• Prototype-based Incremental Few-Shot Segmentation (2020, methodology): “Note that settings similar to iFSS exist in image classification [15, 39, 46], object detection [34] but we are the first to study this setting in semantic segmentation.”
• Domain Generalization: A Survey (2021, background): “range of applications like few-shot classification [201], object detection [203] and image generation [204].”
• DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, background): “Since previous detectors usually require a large amount of annotated data, few-shot detection has attracted more and more interest recently [2, 10, 12, 28, 31, 45, 47, 52, 54].”
• A Survey on Few-Shot Class-Incremental Learning (2023, background): “In the setting of incremental few-shot object detection (iFSD) Perez-Rua et al. (2020), abundant base-class samples and a few novel-class samples are available.”
• Experts Collaboration Learning for Continual Multi-Modal Reasoning (2023, methodology): “[54] investigated the continual few-shot object detection problem, and utilized the meta-learning idea to help the model incorporate new knowledge for registering novel object classes.”