Probabilistic Object Detection: Definition and Evaluation

By David Hall, Feras Dayoub, John Skinner, Haoyang Zhang, Dimity Miller, Peter Corke, Gustavo Carneiro, Anelia Angelova, Niko Sünderhauf. 2018

128 citations recorded by the source corpus.

We introduce Probabilistic Object Detection, the task of detecting objects in images and accurately quantifying the spatial and semantic uncertainties of the detections. Given the lack of methods capable of assessing such probabilistic object detections, we present the new Probability-based Detection Quality measure (PDQ). Unlike AP-based measures, PDQ has no arbitrary thresholds and rewards spatial and label quality, and foreground/background separation quality while explicitly penalising false positive and false negative detections. We contrast PDQ with existing mAP and moLRP measures by evaluating state-of-the-art detectors and a Bayesian object detector based on Monte Carlo Dropout. Our experiments indicate that conventional object detectors tend to be spatially overconfident and thus perform poorly on the task of probabilistic object detection. Our paper aims to encourage the development of new object detection approaches that provide detections with accurately estimated spatial and label uncertainties and are of critical importance for deployment on robots and embodied AI systems in the real world.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability (2019, methodology): “a robot-centric point of view, we employ the ACVR Robotic Vision Challenge dataset [34] which contains simulated data from a domestic robot scenario.”
Towards Open World Object Detection (2021, background): “A dedicated body of work [43, 42, 17] focuses on developing measures of (spatial and semantic) uncertainty in object detectors to reject unknown classes.”
OW-DETR: Open-world Detection Transformer (2021, background): “On the other hand, the works of [7, 12, 22, 23] focus on open-set detection, where new unknown objects encountered during test are to be rejected.”
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving (2020, background): “[132] proposed the Probability-based Detection Quality (PDQ) as a metric to measure the quality of 2D probabilistic object detection on images.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “tion Quality (PDQ) [241] is designed to measure the object”
VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, methodology): “Several works (Dhamija et al., 2020; Miller et al., 2019; 2018; Hall et al., 2020; Deepshikha et al., 2021) used approximate Bayesian methods, such as MC-Dropout (Gal & Ghahramani, 2016) for OOD detection.”