End-to-End Training of Object Class Detectors for Mean Average Precision

By Paul Henderson, Vittorio Ferrari. 2016

339 citations recorded by the source corpus.

We present a method for training CNN-based object class detectors directly using mean average precision (mAP) as the training loss, in a truly end-to-end fashion that includes non-maximum suppresion (NMS) at training time. This contrasts with the traditional approach of training a CNN for a window classification loss, then applying NMS only at test time, when mAP is used as the evaluation metric in place of classification accuracy. However, mAP following NMS forms a piecewise-constant structured loss over thousands of windows, with gradients that do not convey useful information for gradient descent. Hence, we define new, general gradient-like quantities for piecewise constant functions, which have wide applicability. We describe how to calculate these efficiently for mAP following NMS, enabling to train a detector based on Fast R-CNN [1] directly for mAP. This model achieves equivalent performance to the standard Fast R-CNN on the PASCAL VOC 2007 and 2012 datasets, while being conceptually more appealing as the very same model and loss are used at both training and test time.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Non-maximum Suppression (2017, background): “One idea is to include GreedyNMS at training time [33, 12], making the classifier aware of the NMS procedure at test time.”
Learning to Count Objects in Natural Images for Visual Question Answering (2018, background): “Various differentiable variants such as by Azadi et al. (2017), Hosang et al. (2017), and Henderson & Ferrari (2017) exist.”
Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation (2016, neutral): “, [4] and [5] for mAP, [6] for ROC-area).”
Learning With Average Precision: Training Image Retrieval With a Listwise Loss (2019, methodology): “Apart from [24, 25], several relaxations or alternative formulations have been proposed in the literature to allow for the direct optimization of AP [5, 23, 25, 27, 39, 54, 61].”
Imbalance Problems in Object Detection: A Review (2019, methodology): “To make this differentiable, the authors propose a novel error-driven update rule, which performs better than the previous works aiming to include average precision as a training objective [97], [98].”
Spatial Memory for Context Reasoning in Object Detection (2017, background): “Nevertheless, SMN can naturally serve as an encoder of the state in a DRL system to directly optimize average precision [34].”
Object Detection in 20 Years: A Survey (2019, neutral): “Learning-Based NMS: A recent group of NMS improvements that have recently received much attention is learning-based NMS [85], [93], [109], [110], [111], [122].”
Towards Accurate One-Stage Object Detection With AP-Loss (2019, methodology): “Third, approximate gradient methods [31, 9] are proposed for optimizing the AP-loss, which are less efficient and easy to fall into local optimum even for the case of linear models due to the non-convexity and non-quasiconvexity of the AP-loss.”