A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

By Zhaowei Cai, Quanfu Fan, Rogério Schmidt Feris, Nuno Vasconcelos. 2016

1,559 citations recorded by the source corpus.

A unified deep neural network, denoted the multi-scale CNN (MS-CNN), is proposed for fast multi-scale object detection. The MS-CNN consists of a proposal sub-network and a detection sub-network. In the proposal sub-network, detection is performed at multiple output layers, so that receptive fields match objects of different scales. These complementary scale-specific detectors are combined to produce a strong multi-scale object detector. The unified network is learned end-to-end, by optimizing a multi-task loss. Feature upsampling by deconvolution is also explored, as an alternative to input upsampling, to reduce the memory and computation costs. State-of-the-art object detection performance, at up to 15 fps, is reported on datasets, such as KITTI and Caltech, containing a substantial number of small objects.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Feature Pyramid Networks for Object Detection (2016, background): “SSD [22] and MS-CNN [3] predict objects at multiple layers of the feature hierarchy without combining features or scores.”
Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
DeepSetNet: Predicting Sets with Deep Neural Networks (2016, methodology): “To learn the cardinality distribution, we use a gray-scale image as the network input, constructed by superimposing all region proposals and their scores generated by MS-CNN detector (before non-maximum suppression approach).”
Why Should We Add Early Exits to Neural Networks? (2020, methodology): “Apart from Inception, similar approaches have been successfully applied to, among others, face recognition [ 66 ] , object detection [ 15 ] , image segmentation [ 16 ] , pose estimation [ 27 ] , saliency detection [ 43 ] , metric learning [ 76 ] , visual attention prediction [ 74 ] , and super…”
Path Aggregation Network for Instance Segmentation (2018, methodology): “Different from TDM, which took the fused feature map with the highest resolution to pool features, SSD [42], DSSD [13], MS-CNN [5] and FPN [35] assigned proposals to appropriate feature levels for inference.”
EfficientDet: Scalable and Efficient Object Detection (2019, background): “Earlier detectors often directly perform predictions based on the pyramidal feature hierarchy extracted from backbone networks [2, 20, 28].”
StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection (2017, methodology): “Similar to SSD, MS-CNN [2] also uses multiple feature maps for prediction and they newly introduced deconvolution layer to increase the resolution of feature maps.”
CoupleNet: Coupling Global Structure with Local Parts for Object Detection (2017, methodology): “[2] collected the context by padding the proposals for pedestrian and car detection.”
Multi-view 3D Object Detection Network for Autonomous Driving (2016, methodology): “MS-CNN [1] Mono 90.”
CornerNet: Detecting Objects as Paired Keypoints (2018, background): “…sub-category information (Xiang et al., 2016), generating object proposals at multiple scales with more contextual information (Bell et al., 2016; Cai et al., 2016; Shrivastava et al., 2016; Lin et al., 2016), selecting better features (Zhai et al., 2017), improving speed (Li et al., 2017),…”
DSSD : Deconvolutional Single Shot Detector (2017, methodology): “MS-CNN [2] applies deconvolution on multiple layers of a ConvNet to increase feature map resolution before using the layers to learn region proposals and pool features.”
Single-Shot Refinement Neural Network for Object Detection (2017, methodology): “After that, numerous effective techniques are proposed to further improve the performance, such as architecture diagram [5, 25, 55], training strategy [31, 41, 48, 50], contextual reasoning [1, 14, 40, 51] and multiple layers exploiting [3, 24, 26, 42].”
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification (2021, methodology): “In the context of CNNs, multi-scale feature representations have been used for detection and recognition of objects at multiple scales [4, 22, 44, 26], as well as to speed up neural networks in Big-Little Net [5] and OctNet [6].”
CenterNet: Keypoint Triplets for Object Detection (2019, background): “, a [52, 22] focus on the architecture design, a [1, 10, 37, 47] focus on the contextual relationship, and a [23, 3] focus on the multi-scale unification.”
Libra R-CNN: Towards Balanced Learning for Object Detection (2019, methodology): “Other methods focus on cascade procedures [24], duplicate removal [14, 13], multi-scales [2, 1, 31, 30], adversarial learning [37] and more contextual [36].”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “There are related multi-scale networks for classification and segmentation [5, 8, 72, 78, 29, 73, 53, 54, 23, 80, 53, 51, 18].”