IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection

By Liyang Liu, Zhanghui Kuang, Yimin Chen, Jing-Hao Xue, Wenming Yang, Wayne Zhang. 2020

76 citations recorded by the source corpus.

Elastic weight consolidation (EWC) has been successfully applied for general incremental learning to overcome the catastrophic forgetting issue. It adaptively constrains each parameter of the new model not to deviate much from its counterpart in the old model during fine-tuning on new class data sets, according to its importance weight for old tasks. However, the previous study demonstrates that it still suffers from catastrophic forgetting when directly used in object detection. In this article, we show EWC is effective for incremental object detection if with critical adaptations. First, we conduct controlled experiments to identify two core issues why EWC fails if trivially applied to incremental detection: 1) the absence of old class annotations in new class images makes EWC misclassify objects of old classes in these images as background and 2) the quadratic regularization loss in EWC easily leads to gradient explosion when balancing old and new classes. Then, based on the abovementioned findings, we propose the corresponding solutions to tackle these issues: 1) utilize pseudobounding box annotations of old classes on new data sets to compensate for the absence of old class annotations and 2) adopt a novel Huber regularization instead of the original quadratic loss to prevent from unstable training. Finally, we propose a general EWC-based incremental object detection framework and implement it under both Fast R-CNN and Faster R-CNN, showing its flexibility and versatility. In terms of either the final performance or the performance drop with respect to the upper bound of joint training on all seen classes, evaluations on the PASCAL VOC and COCO data sets show that our method achieves a new state of the art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dipam Goswami
2. Liu Xialei
3. Xin Ning
4. Scott Sanner
5. Zheda Mai
6. Weijun Li
7. Liu Yu-yang
8. David Quispe

In Computer Vision and Pattern Recognition

1. Dipam Goswami
2. Hyunwoo J. Kim
3. Scott Sanner
4. Zheda Mai
5. Ruiwen Li
6. Liu Xialei
7. Xin Ning
8. Weijun Li

In Radiology, Nuclear Medicine and Imaging

1. Scott Sanner
2. Zheda Mai
3. Hyunwoo J. Kim
4. Jihwan Jeong
5. Joost van de Weijer

In Meta & Continual Learning

1. Liu Xialei
2. Scott Sanner
3. Zheda Mai
4. Liu Yu-yang
5. David Quispe
6. Hyunwoo J. Kim
7. Jihyeon Kim
8. Mang Wang

Citations in the corpus

• Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation (2022, methodology): “Nevertheless, the nature of these methods are designed using feature-based knowledge distillation [6], response-based method [12] has not been explored in IOD [25] yet.”
• Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “The Huber regularization proposed lately could be a possible remedy [110].”
• MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, neutral): “Recently, there has been a large body of research in incremental learning [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43].”
• A Survey on Few-Shot Class-Incremental Learning (2023, background): “There are also some researchers (Chen et al., 2019; Liu et al., 2020b; Shmelkov et al., 2017) who study the problem of incremental target detection from the perspective of IL.”
• Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, methodology): “EWC [31] has been adapted by [38] for object detection.”
• SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection (2024, methodology): “Initial researches [1, 11, 35, 60] in class incremental object detection (CIOD) extended image classification methods This CVPR paper is the Open Access version, provided by the Computer Vision Foundation.”