Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation

By Mang Wang, Hangjie Yuan. 2022

154 citations recorded by the source corpus.

Traditional object detectors are ill-equipped for incremental learning. However, fine-tuning directly on a well-trained detection model with only new data will lead to catastrophic forgetting. Knowledge distillation is a flexible way to mitigate catastrophic forgetting. In Incremental Object Detection (IOD), previous work mainly focuses on distilling for the combination of features and responses. However, they under-explore the information that contains in responses. In this paper, we propose a response-based incremental distillation method, dubbed Elastic Response Distillation (ERD), which focuses on elastically learning responses from the classification head and the regression head. Firstly, our method transfers category knowledge while equipping student detector with the ability to retain localization information during incremental learning. In addition, we further evaluate the quality of all locations and provide valuable responses by the Elastic Response Selection (ERS) strategy. Finally, we elucidate that the knowledge from different responses should be assigned with different importance during incremental distillation. Extensive experiments conducted on MS COCO demonstrate our method achieves state-of-the-art result, which substantially narrows the performance gap towards full training. Code is available at https://github.com/Hi-FT/ERD.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Jun Zhu
6. Qiwen Wang
7. Ziwei Liu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Da-Wei Zhou
5. Hang Su
6. Ziwei Liu
7. Liyuan Wang
8. Xingxing Zhang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Jun Zhu
8. Qiwen Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “The idea of knowledge distillation is then introduced to other detection frameworks [283], [284], [285].”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “To solve this problem, they proposed to separate parameters into global and task-specific parameters, whereby each client can obtain a weighted combination of the task-specific parameters of the other clients for knowledge transfer.”
• Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, background): “architecture and performs distillation on the intermediate features [7, 22, 40, 48, 60, 66], the region proposal network [7, 48, 66] and head layers [17].”
• Consistent Prompting for Rehearsal-Free Continual Learning (2024, unknown): “As a prevalent regularization method, knowledge distillation [4, 8, 47] is widely adopted to transfer the retained knowledge in the previous model (as a teacher) to the current student model.”
• Deep Class-Incremental Learning: A Survey (2023, unknown): “, semantic segmentation [164], [165], [166], [167], [174], object detection [175], [176], [177], [178], [179], [180], [181], video understanding [95], [133], medical surgery [182], [183], language model [184], [185], [186], [187], language-vision model [188], [189], etc.”
• Class-Incremental Learning: A Survey (2023, background): “…into other tasks, e.g. , semantic segmentation [193], [238], [239], [240], [241], object detection [137], [242], [243], [244], [245], [246], [247], video understanding [91], [161], medical surgery [248], [249], language model [250], [251], [252], [253], language-vision model [217], [254],…”
• ModelScope Text-to-Video Technical Report (2023, background): “Despite initializing the spatial part of ModelScopeT2V with Stable Diffusion [46], training solely on videotext paired datasets can hinder semantic diversity and lead to catastrophic forgetting of image-domain expertise during training [34, 11, 30, 12].”
• SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection (2024, methodology): “[11, 28, 33, 43] extract from CL-DETR paper.”