OW-DETR: Open-world Detection Transformer

By Akshita Gupta, Sanath Narayan, K. J. Joseph, Salman Hameed Khan, Fahad Shahbaz Khan, Mubarak Shah. 2021

275 citations recorded by the source corpus.

Open-world object detection (OWOD) is a challenging computer vision problem, where the task is to detect a known set of object categories while simultaneously identifying unknown objects. Additionally, the model must incrementally learn new classes that become known in the next training episodes. Distinct from standard object detection, the OWOD setting poses significant challenges for generating quality candidate proposals on potentially unknown objects, separating the unknown objects from the background and detecting diverse unknown objects. Here, we introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection. The proposed OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification and objectness scoring to explicitly address the aforementioned OWOD challenges. Our OW-DETR explicitly encodes multi-scale contextual information, possesses less inductive bias, enables knowledge transfer from known classes to the unknown class and can better discriminate between unknown objects and background. Comprehensive experiments are performed on two benchmarks: MS-COCO and PASCAL VOC. The extensive ablations reveal the merits of our proposed contributions. Further, our model out-performs the recently introduced OWOD approach, ORE, with absolute gains ranging from 1.8% to 3.3% in terms of unknown recall on MS-COCO. In the case of incremental object detection, OW-DETR outperforms the state-of-the-art for all settings on PASCAL VOC. Our code is available at https://github.com/akshitac8/OW-DEtr.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Energy-based Latent Aligner for Incremental Learning (2022, methodology): “In this regard, knowledge distillation [19] has been used extensively to enforce explicit regularization in incremental classiﬁcation [7,28,41] and object detection [15,21,46] settings.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “In particular, OW-DETR [287] adopts the DETR as the base detector and proposes several improvements, including attention-driven pseudo-labeling, novelty classiﬁcation, and objectness scoring.”
Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, methodology): “Our improvements over the methods storing exemplars [21,27,28] confirm the importance of the box replay for IOD.”
SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection (2024, methodology): “CIOD has progressed from primarily employing CNN-based methods [1, 11, 28, 36, 43, 60] to also incorporating Transformer-based approaches [10, 15, 24, 25, 37].”