An End-to-End Architecture for Class-Incremental Object Detection with Knowledge Distillation

By Yu Hao, Yanwei Fu, Yu-Gang Jiang, Qi Tian. 2019

94 citations recorded by the source corpus.

Recent efforts have been made on continuously learning concepts over time, i.e., class-incremental learning, which, however, is still an open question to the task of object detection. For the first time, this paper systematically studies the task of learning an end-to-end class-incremental object detection model. Remarkably, this work tailors Faster R-CNN and proposes a Class-Incremental Faster R-CNN (CIFRCN) model that can dynamically add new classes by only using a few labeled images of new objects. Particularly, the domain of "foreground" in a Region Proposal Network is expanded to generate accurate bounding box proposals, and the classifier in Fast R-CNN is enlarged using knowledge distillation to accurately classify the proposals. Extensive experiments on two challenging datasets are conducted to demonstrate the efficacy of the proposed model for adding new objects in a class-incremental way, whereas other detectors quickly fail.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “The idea of knowledge distillation is then introduced to other detection frameworks, such as CenterNet [508] (SID [328]), RetinaNet [259] (RILOD [247]), GFLV1 [250] (ERD [114]) and Faster R-CNN [358] (CIFRCN [152], Faster ILOD [327], DMC [493], BNC [96], IOD-ML [222]).”
Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, methodology): “Built upon this first work, CIFRCN (Hao et al. 2019) additionally distills the intermediate features of RPN by adopting Faster R-CNN.”
Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object Detection (2021, neutral): “Recently, several incremental learning methods [40, 27, 4, 10, 20, 31] are proposed to overcome catastrophic forgetting in the object detection task.”
Towards Open World Object Detection (2021, background): “The prevalent approaches [61, 28, 18, 7] use knowledge distillation [21]”
SID: Incremental Learning for Anchor-Free Object Detection via Selective and Inter-Related Distillation (2020, methodology): “Hao et al. (2019) proposed an incremental method based on Faster R-CNN (Ren et al., 2015).”
Incremental Object Detection via Meta-Learning (2020, positive): “All of the methods proposed so far [41,22,13,5] use a regularization technique called knowledge distillation [16] to address catastrophic forgetting.”
Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNN (2020, methodology): “Unlike what was assumed in some previous literature (Hao et al., 2019; Shmelkov et al., 2017), we see that, the performance of Faster RCNN is not largely deteriorated in the incremental settings where the annotations of old classes are not provided in the new data.”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, positive): “[89] extend Faster-RCNN [90] with knowledge distillation.”
RODEO: Replay for Online Object Detection (2020, neutral): “While streaming object detection has not been explored, there has been some work on object detection in the continual (batch) learning paradigm [14, 29, 50, 51].”
SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection (2024, methodology): “In CIL, we can cluster the main methods into knowledge distillation [16, 33, 38, 61], and re-play [5–7, 12, 17, 22, 47, 50, 57, 58, 66, 67] in general.”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, unknown): “Due to its potential usefulness and effectiveness, continual learning has attracted increasing interests in recent years for a wide range of vision applications, such as image classi-ﬁcation [38]–[40], object detection [41], [42], and semantic segmentation [43], [44].”
Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, methodology): “While most studies in incremental learning are based on image classification [2, 31, 34, 50], recently it has been studied in the context of object detection [7, 9, 22, 48, 55] and semantic segmentation [13, 20, 64].”
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning (2020, background): “In recent years, incremental learning has become one of the most critical yet challenging directions in a broad spectrum of application domains, including image classification [24,35], object detection [39,10] and semantic segmentation [30,5].”
MetaSearch: Incremental Product Search via Deep Meta-Learning (2020, methodology): “They also [22] introduced an end-to-end class-incremental object detection model by using only a few labeled images of new objects, which alleviated the catastrophic forgetting problem and achieved excellent performance.”
IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection (2020, methodology): “To compare fairly with the recent method CIFRCN [23], we follow their settings and split the 20 classes in the VOC dataset to 4 groups (a, b, c, and d) with 5 classes in each group.”
Incremental Detection of Remote Sensing Objects With Feature Pyramid and Knowledge Distillation (2020, methodology): “In comparative experiments, two state-of-the-art incremental object detection methods in [19] and [20] are used as reference methods.”