Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

By Kaiming He, X. Zhang, Shaoqing Ren. 2014

12,661 citations recorded by the source corpus.

Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224<inline-formula><tex-math>$\times$ </tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq1-2389824.gif"/></alternatives></inline-formula>224) input image. This requirement is “artificial” and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, “spatial pyramid pooling”, to eliminate the above requirement. The new network structure, called SPP-net, can generate a fixed-length representation regardless of image size/scale. Pyramid pooling is also robust to object deformations. With these advantages, SPP-net should in general improve all CNN-based image classification methods. On the ImageNet 2012 dataset, we demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures despite their different designs. On the Pascal VOC 2007 and Caltech101 datasets, SPP-net achieves state-of-the-art classification results using a single full-image representation and no fine-tuning. The power of SPP-net is also significant in object detection. Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training the detectors. This method avoids repeatedly computing the convolutional features. In processing test images, our method is 24-102 <inline-formula><tex-math>$\times$</tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq2-2389824.gif"/> </alternatives></inline-formula> faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007. In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our methods rank #2 in object detection and #3 in image classification among all 38 teams. This manuscript also introduces the improvement made for this competition.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Residual Learning for Image Recognition (2015, background): “Many other nontrivial visual recognition tasks [8, 12, 7, 32, 27] have also”
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015, methodology): “Adaptively-sized pooling (SPP) [7] on shared conv feature maps is proposed for efﬁcient region-based object detection [7, 16] and semantic segmentation [2].”
Fast R-CNN (2015, methodology): “Unfortunately, the approach of [10] restricts the back-propagation of errors through the network during training, potentially limiting accuracy.”
Object Detection Networks on Convolutional Feature Maps (2015, methodology): “We use a 172 ZF model released by [12]2.”
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, methodology): “In the feature map, each 14×14 window is pooled using the SPP layer [12].”
Fully convolutional networks for semantic segmentation (2014, background): “[17] discard the nonconvolutional portion of classification nets to make a feature extractor.”
Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, background): “The resulting output feature map is a class score map with the number of channels equal to the number of classes, and the variable spatial resolution, dependent on the input image size.”
ImageNet Large Scale Visual Recognition Challenge (2014, neutral): “of China∓ Kaiming He†, Xiangyu Zhang‡, Shaoqing Ren∓, Jian Sun†(He et al., 2014)”
Mask R-CNN (2017, methodology): “So even though RoIWarp also adopts bilinear resampling motivated by [22], it performs on par with RoIPool as shown by experiments (more details in Table 2c), demonstrating the crucial role of alignment.”
Rethinking ImageNet Pre-Training (2018, background): “As modern object detectors [9, 15, 8, 36, 35, 28, 26, 13] evolved under the pre-training paradigm, the belief that training from scratch is non-trivial became conventional wisdom.”
Relation Networks for Object Detection (2017, methodology): “The state-of-the-art object detection methods [24, 18, 38, 9, 32, 10, 23] mostly follow the region based paradigm since it is established in the seminal work R-CNN [19].”
Feature Pyramid Networks for Object Detection (2016, methodology): “SPPnet [15] demonstrated that such region-based detectors could be applied much more efﬁciently on feature maps extracted on a single image scale.”
Cascade R-CNN: High Quality Object Detection and Instance Segmentation (2019, background): “SPP-Net [25] and Fast R-CNN [19] introduced the idea of region-wise feature extraction, enabling the sharing of the”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, methodology): “In a prevalent family of instance-aware semantic segmentation approaches [7, 16, 8], it is achieved by adopting different types of sub-networks in three stages: 1) an FCN is applied on the whole image to generate intermediate and shared feature maps; 2) from the shared feature maps, a pooling layer ”
SSD: Single Shot MultiBox Detector (2015, methodology): “Fast R-CNN [6] extends SPPnet so that it can fine-tune all layers end-toend by minimizing a loss for both confidences and bounding box regression, which was first introduced in MultiBox [7] for learning objectness.”
You Only Look Once: Unified, Real-Time Object Detection (2015, neutral): “[5] [37] [30] [14] [17] [27] However, only Sadeghi et al.”