Diagnosing Error in Object Detectors

By Derek Hoiem, Yodsawalai Chodpathumwan, Qieyun Dai. 2012

560 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, positive): “We use the tool of Hoiem et al. [15] to analyse the detection errors of our system.”
Object Detection Networks on Convolutional Feature Maps (2015, methodology): “The total number of samples in each disk is the same and equal to the total number of ground-truth labels [30].”
ImageNet Large Scale Visual Recognition Challenge (2014, background): “As described in (Hoiem et al., 2012), smaller objects tend to be signiﬁcantly more diﬃcult to local-ImageNet ize.”
Microsoft COCO: Common Objects in Context (2014, background): “…struggle to recognize objects otherwise – in the We introduce a large, richly-annotated dataset comprised of images depicting complex everyday scenes of common objects in their natural context. background, partially occluded, amid clutter [13] – re-ﬂecting the composition of actual everyday scenes.”
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, methodology): “We also discuss the trade-offs involved in training detection SVMs rather than simply using the outputs from the final softmax layer of the fine-tuned CNN.”
Webly Supervised Learning of Convolutional Networks (2015, methodology): “0 50 60 70 80 90 100 Loc Sim Oth total false positives percentage of each type vehicles 25 50 100 200 400 800 1600 3200 0 10 20 30 40 50 60 70 80 90 100 Loc Sim Oth Figure 6. Diagnosis analysis using [25] for better understanding of the failure modes of our webly supervised pipeline. Please see top ”
Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “Following standard error diagnosis studies for object detectors [38], we show the distribution of false positive (FP) errors, considering the following types: localiza-”
SSD: Single Shot MultiBox Detector (2015, methodology): “In addition, we used the detection analysis tool from [9] to understand what characteristics affects SSD the most.”
You Only Look Once: Unified, Real-Time Object Detection (2015, background): “[19] For each category at test time we look at the top N predictions for that category.”
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, methodology): “Similarly, we show a size breakdown for VOC2007 test in Figure 5 using Hoiem’s toolkit for diagnosing errors [15], and see similarly large improvements on this dataset as well.”
What Makes for Effective Detection Proposals? (2015, background): “Evaluating (class agnostic) detection proposals is quite different from traditional class-specific detection [55] since most metrics (class confusion, background confusion, precision, etc.”
Simultaneous Detection and Segmentation (2014, methodology): “We can also measure impact in this way for other error modes: for instance, false positives on objects of other similar categories, or on background [18].”
LVIS: A Dataset for Large Vocabulary Instance Segmentation (2019, background): “This ordering shows that for a fixed classifier, AP tends to be lower for rarer categories, which is an intuitive and wellknown trend (see [14] §3.”
A MultiPath Network for Object Detection (2016, background): “75 [.706] C50 [.768] Loc [.784] Sim [.816] Oth [.944] BG [1.00] FN (i) large Figure 6: Detailed analysis of detector performance on unseen COCO validation images at select settings (plots in style of [16] generated by COCO API code). (a) Removing localization errors would lead to an AP10 of 58.8 on ”
Visualizing and Understanding Recurrent Networks (2015, methodology): “This is a standard smoothing method for language models Huang et al. (2001). All models were trained using the popular KenLM software package Heafield et al.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, background): “visual question answering [3]), the goal of transparency and explanations is to identify the failure modes [1, 17], thereby helping researchers focus their efforts on the most fruitful research directions.”
Analyzing the Behavior of Visual Question Answering Models (2016, methodology): “(Hoiem et al., 2012) provided analysis tools to facilitate detailed and meaningful investigation of object detector performance.”
Detect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts (2014, background): “[15], where each instance is assigned to a size category, depending on its percentile size: extra-small (XS: bottom 10%); small (S: next 20%); medium (M: next 40%); large (L: next 20%); extra-large (XL: next 10%).”