The Overlooked Elephant of Object Detection: Open Set
By Akshay Raj Dhamija, Manuel Günther, Jonathan Ventura, Terrance E. Boult. 2020
199 citations recorded by the source corpus.
Even though object detection is a popular area of research that has found considerable applications in the real world, it has some fundamental aspects that have never been formally discussed and experimented. One of the core aspects of evaluating object detectors has been the ability to avoid false detections. While major datasets like PASCAL VOC or MSCOCO extensively test the detectors on their ability to avoid false positives, they do not differentiate between their closed-set and open-set performance. Despite systems being trained to reject everything other than the classes of interest, unknown objects from the open world end up being incorrectly detected as known objects, often with very high confidence. This paper is the first to formalize the problem of open-set object detection and propose the first open-set object detection protocol. Moreover, the paper provides a new evaluation metric to analyze the performance of some state-of-the-art detectors and discusses their performance differences.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Anomaly & Outlier Detection
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Zhe Gan
- Mohit Bansal
- Chunyuan Li
- Jianwei Yang
- Lijuan Wang
- Bernt Schiele
- Fahad Shahbaz Khan
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Bernt Schiele
In Anomaly & Outlier Detection
- Fahad Shahbaz Khan
- Vineeth Nallure Balasubramanian
- K. J. Joseph
- Xuefeng Du
- Yixuan Li
- Salman Hameed Khan
- Deva Ramanan
- Han-Jia Ye
Citations in the corpus
- Grounded Language-Image Pre-training (2021, background): “Our setting differs from zero-shot detection [1, 12, 14, 46, 47, 68], where some categories are defined as unseen/rare and not present in the training set.”
- Towards Open World Object Detection (2021, methodology): “[8] has found that it has better open set performance when compared against one-stage RetinaNet detector [31] and objectness based YOLO detector [52].”
- Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation (2021, background): “[8] discussed the difficulties in open-world object detection.”
- FindIt: Generalized Localization with Natural Language Queries (2022, background): “Many recent approaches have started to study the open-set and zero-shot settings [15,2,84,98,20].”
- OW-DETR: Open-world Detection Transformer (2021, background): “On the other hand, the works of [7, 12, 22, 23] focus on open-set detection, where new unknown objects encountered during test are to be rejected.”
- Opening up Open World Tracking (2021, methodology): “We build on intuition [17, 19, 56] that learned object proposal mechanisms, such as Region Proposal Net-”
- VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, methodology): “Several works (Dhamija et al., 2020; Miller et al., 2019; 2018; Hall et al., 2020; Deepshikha et al., 2021) used approximate Bayesian methods, such as MC-Dropout (Gal & Ghahramani, 2016) for OOD detection.”
- Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, background): “[133], where a novel open-set protocol and metrics are proposed.”
- A Training-Free Approach to Long-Tail Object Detection for Autonomous Driving (2026, unknown): “Open-set recognition in autonomous driving has been studied from multiple angles: detecting unknown objects, identifying out-of-distribution inputs, modeling uncertainty, and handling long-tail distributions [5, 18].”