Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism

By Binbin Yang, Xi Deng, Han Shi, Changlin Li, Gengwei Zhang, Hang Xu, Shen Zhao, Liang Lin, Xiaodan Liang. 2022

46 citations recorded by the source corpus.

Continual learning is a challenging real-world problem for constructing a mature AI system when data are provided in a streaming fashion. Despite recent progress in continual classification, the researches of continual object detection are impeded by the diverse sizes and numbers of objects in each image. Different from previous works that tune the whole network for all tasks, in this work, we present a simple and flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTing mechAnism (ROSETTA). Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. In this way, various knowledge can be successively memorized by storing their corresponding sub-model weights in this system. To make ROSETTA automatically determine which experience is available and useful, a prototypical task correlation guided Gating Diversity Controller (GDC) is introduced to adaptively adjust the diversity of gates for the new task based on class-specific prototypes. GDC module computes class-to-class correlation matrix to depict the cross-task correlation, and hereby activates more exclusive gates for the new task if a significant domain gap is observed. Comprehensive experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance on both task-based and class-based continual object detection.11Codes are available at: https://github.com/dkxocl/ROSSETA.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yunchao Wei
2. Guoliang Kang
3. Gengwei Zhang
4. Liyuan Wang
5. Ling Chen
6. Yi Yang
7. Junsuk Choe
8. Thomas S. Huang

In Computer Vision and Pattern Recognition

1. Yunchao Wei
2. Guoliang Kang
3. Gengwei Zhang
4. Liyuan Wang
5. Ling Chen
6. Yongjun Zhang
7. Bo Dang
8. Xin Guo

Citations in the corpus

• SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model (2023, background): “Representative strategies include regularization-based approaches [19, 1, 51, 25, 41, 7], which preserve the old model and selectively stabilize changes of parameters or predictions; replaybased approaches [49, 30, 3, 44, 40, 39], which approximate and recover the previously-learned data distributio”
• Continual Detection Transformer for Incremental Object Detection (2023, unknown): “Unsurprisingly, forgetting also affects object detection [2, 12, 20, 37, 44, 50, 54].”
• SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (2023, background): “Recently, Foundation Model emerges as a pre-trained generic model that excels in a wide range of downstream tasks [82, 88].”