Continual Detection Transformer for Incremental Object Detection

By Yaoyao Liu, Bernt Schiele, Andrea Vedaldi, Christian Rupprecht. 2023

122 citations recorded by the source corpus.

Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories. As other incremental settings, IOD is subject to catastrophic forgetting, which is often addressed by techniques such as knowledge distillation (KD) and exem-plar replay (ER). However, KD and ER do not work well if applied directly to state-of-the-art transformer-based object detectors such as Deformable DETR [59] and UP-DETR [9]. In this paper, we solve these issues by proposing a ContinuaL DEtection TRansformer (CL-DETR), a new method for transformer-based IOD which enables effective usage of KD and ER in this context. First, we introduce a Detector Knowledge Distillation (DKD) loss, focusing on the most informative and reliable predictions from old versions of the model, ignoring redundant background predictions, and ensuring compatibility with the available ground-truth labels. We also improve ER by proposing a calibration strategy to preserve the label distribution of the training set, therefore better matching training and testing statistics. We conduct extensive experiments on COCO 2017 and demonstrate that CL-DETR achieves state-of-the-art results in the IOD setting.11Code: https://1yy.mpi-inf.mpg.de/CL-DETR/

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, background): “Future research should explore the implications of the foreground shift in incremental semantic segmentation and extend our approach to popular transformer methods [42].”
Class-Incremental Learning: A Survey (2023, background): “Most of these works are based on conventional detectors, while CL-DERT [267] finds directly applying knowledge distillation or data replay works poorly on transformer-based detectors ( e.g. , Deformable DETR [268] and UP-DETR [269]).”
SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection (2024, methodology): “Based on the prediction mechanism introduced in [15, 37], D-DETR utilizes learned object queries in the decoder.”