A Survey on Performance Metrics for Object-Detection Algorithms

By Rafael Padilla, Sergio L. Netto, Eduardo A. B. da Silva. International Conference on Systems, Signals, and Image Processing, 2020

1,320 citations recorded by the source corpus.

This work explores and compares the plethora of metrics for the performance evaluation of object-detection algorithms. Average precision (AP),for instance, is a popular metric for evaluating the accuracy of object detectors by estimating the area under the curve (AUC) of the precision × recall relationship. Depending on the point interpolation used in the plot, two different AP variants can be defined and, therefore, different results are generated. AP has six additional variants increasing the possibilities of benchmarking. The lack of consensus in different works and AP implementations is a problem faced by the academic and scientific communities. Metric implementations written in different computational languages and platforms are usually distributed with corresponding datasets sharing a given bounding-box description. Such projects indeed help the community with evaluation tools, but demand extra work to be adapted for other datasets and bounding-box formats. This work reviews the most used metrics for object detection detaching their differences, applications, and main concepts. It also proposes a standard implementation that can be used as a benchmark among different datasets with minimum adaptation on the annotation files.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1187e4a921beee82ebd9f5f99a79f20397d3549d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

The Computational Limits of Deep Learning (2020, methodology): “Face detection is measured using the metric Average Precision (AP) which is a popular metric for evaluating the accuracy of object detectors by estimating the area under the curve (AUC) of the precision × recall relationship [137].”
Visible and Thermal Image-Based Trunk Detection with Deep Learning for Forestry Mobile Robotics (2021, methodology): “Recall measures how many objects are detected, and depends on the True Positives and False Negatives, as can be seen in (2).”
Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-50 for medical face mask detection (2020, neutral): “The average precision (AP) (Padilla, Netto, & da Silva, 2020) combines recall and precision as shown in equation (6) to evaluate the ability of the detector to find all relevant objects and the ability of the detector to detect objects correctly.”
A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit (2021, methodology): “Our previous work [23] contributed to the research community in this direction, by presenting a tool which reads ground-truth and detected bounding boxes in a closed format and evaluates the detections using the average precision (AP) and mean average precision (mAP) metrics, as required in the PASC”