Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors

By Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara Balan, Alireza Fathi, Ian S. Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama and 1 more. 2016

2,700 citations recorded by the source corpus.

The goal of this paper is to serve as a guide for selecting a detection architecture that achieves the right speed/memory/accuracy balance for a given application and platform. To this end, we investigate various ways to trade accuracy for speed and memory usage in modern convolutional object detection systems. A number of successful systems have been proposed in recent years, but apples-toapples comparisons are difficult due to different base feature extractors (e.g., VGG, Residual Networks), different default image resolutions, as well as different hardware and software platforms. We present a unified implementation of the Faster R-CNN [30], R-FCN [6] and SSD [25] systems, which we view as meta-architectures and trace out the speed/accuracy trade-off curve created by using alternative feature extractors and varying other critical parameters such as image size within each of these meta-architectures. On one extreme end of this spectrum where speed and memory are critical, we present a detector that achieves real time speeds and can be deployed on a mobile device. On the opposite end in which accuracy is critical, we present a detector that achieves state-of-the-art performance measured on the COCO detection task.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Massive Exploration of Neural Machine Translation Architectures (2017, methodology): “While sweeping across large hyperparameter spaces is common in Computer Vision (Huang et al., 2016b), such exploration would be prohibitively expensive for NMT models, limiting researchers to well-established architectures and hyperparameter choices.”
Mask R-CNN (2017, methodology): “Mask R-CNN using ResNet-101FPN outperforms the base variants of all previous state-ofthe-art models, including the single-model variant of GRMI [21], the winner of the COCO 2016 Detection Challenge.”
Do Better ImageNet Models Transfer Better? (2018, methodology): “[6, 61, 32, 30, 27]), they have never been systematically explored across network architectures.”
Relation Networks for Object Detection (2017, background): “Recent years have witnessed significant progress in object detection using deep convolutional neutral networks (CNNs) [27].”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, methodology): “For COCO [26], we use a held-out 8000 images from the standard ‘val’ set as our validation set, we refer to it as ‘minival∗’, the same set of images was used by [18].”
Natural Adversarial Examples (2019, background): “Research on the ImageNet [10] benchmark has led to numerous advances in classification [36], object detection [34], and segmentation [21].”
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning (2018, methodology): “Based on the ﬁndings of Huang et al. (2016), we use Inception-ResNet-v2 (Szegedy et al., 2016) for image-feature extraction, which confers optimization beneﬁts via residual connections and computationally efﬁcient Inception units.”
Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “6 G-RMI [19] trainval35k Ensemble of Five Models 41.”
Cascade R-CNN: High Quality Object Detection and Instance Segmentation (2019, neutral): “G-RMI [28], the recent Deformable R-FCN [8], RetinaNet [36], Mask R-CNN [24], RelationNet [27], DetNet [34], CornerNet [32], etc.”
Objects as Points (2019, background): “Nonetheless, over the past ﬁve years [19], this idea has achieved good empirical success [12,21,25,26,31,35,47,48,56,62,63].”
MegDet: A Large Mini-Batch Object Detector (2017, methodology): “ in RetinaNet [24], in just two years. The improvements are mainly due to better backbone network [16], new detection framework [31], novel loss design [24], improved pooling method [5,14], and so on [19]. A recent trend on CNN-based image classiﬁcation uses very large min-batch size to signiﬁcantly”
Cascade R-CNN: Delving Into High Quality Object Detection (2017, neutral): “This includes the single-model entries of the COCO challenge winners in 2015 and 2016 (Faster R-CNN+++ [16], and G-RMI [17]), and the very recent Deformable R-FCN [5], RetinaNet [22] and Mask R-CNN [14].”
Focal Loss for Dense Object Detection (2017, background): “2 Faster R-CNN by G-RMI [33] Inception-ResNet-v2 [38] 34.”
On Network Design Spaces for Visual Recognition (2019, background): “wall-clock speed [12], can easily be integrated into our paradigm.”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, methodology): “We report results for MobileNet trained for object detection on COCO data based on the recent work that won the 2016 COCO challenge [10].”
Learning Transferable Architectures for Scalable Image Recognition (2017, methodology): “To further validate the performance of the family of NASNet-A networks, we test whether object detection systems derived from NASNet-A lead to improvements in object detection [18].”