Deformable part models are convolutional neural networks

By Ross B. Girshick, Forrest N. Iandola, Trevor Darrell, Jitendra Malik. 2014

469 citations recorded by the source corpus.

Deformable part models (DPMs) and convolutional neural networks (CNNs) are two widely used tools for visual recognition. They are typically viewed as distinct approaches: DPMs are graphical models (Markov random fields), while CNNs are “black-box” non-linear classifiers. In this paper, we show that a DPM can be formulated as a CNN, thus providing a synthesis of the two ideas. Our construction involves unrolling the DPM inference algorithm and mapping each step to an equivalent CNN layer. From this perspective, it is natural to replace the standard image features used in DPMs with a learned feature extractor. We call the resulting model a DeepPyramid DPM and experimentally validate it on PASCAL VOC object detection. We find that DeepPyramid DPMs significantly outperform DPMs based on histograms of oriented gradients features (HOG) and slightly outperforms a comparable version of the recently introduced R-CNN detection system, while running significantly faster.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Object Detection Networks on Convolutional Feature Maps (2015, methodology): “The mAP on PASCAL VOC 2007 is 45-47% [8], [9], [10], [11] for hybrid methods, and is 47% for R-CNN that just uses SVM on the last convolutional layer.”
Mask R-CNN (2017, methodology): “We run the box prediction branch on these proposals, followed by non-maximum suppression [14].”
Instance-Sensitive Fully Convolutional Networks (2016, methodology): “In [28], the AR metrics have been shown to be more correlated to the detection accuracy (when used with downstream classiﬁers [2,21]) than traditional metrics for evaluating proposals.”
Deformable Convolutional Networks (2017, methodology): “Deformable RoI pooling is the first to learn pooling regions end-to-end in CNNs.”
Conditional Random Fields as Recurrent Neural Networks (2015, methodology): “as used for human pose estimation. Similarly, Ross et al . [47] and Stoyanov et al . [54] showed how back-propagation through belief propagation can be used to optimize model parameters. Ross et al . [21], in particular proposes an approach based on learning messages. Many of these ideas can be trac”
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size (2016, methodology): “ImageNettrained DNNs have also been applied to a number of applications pertaining to autonomous driving, including pedestrian and vehicle detection in images [12, 7] and videos [2], as well as segmenting the shape of the road [1].”
Recent advances in convolutional neural networks (2015, background): “Since 2012, the huge success of CNNs in ImageNet challenge [8] rekindles interest in CNN-based object detection [197].”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “tations of the places can overcome the challenges for various appearance and viewpoints in the same scene. 4) End-to-end: An essential reason to perform DL with hierarchical convolutional operations (Girshick et al., 2015; Zheng et al., 2015; Engelcke et al., 2016) is to build the endto-end model. T”
When Kernel Methods Meet Feature Learning: Log-Covariance Network for Action Recognition From Skeletal Data (2017, background): “Also, sometimes the connections are just a re-interpretation of existing paradigms [9], which is theoretically interesting but not advantageous in terms of learning better models.”
Higher Order Conditional Random Fields in Deep Neural Networks (2015, methodology): “ higher order potentials in an end-to-end trainable CNN. Graphical models have been used with CNNs in other areas besides semantic segmentation, such as in poseestimation [42], deformable part models [14], and group activity recognition [8]. However, the nature of models used in these works is subst”
Viewpoints and keypoints (2014, methodology): “[12] argued that DPMs could also be thought as a specific instantiation of CNNs and therefore training an end-to-end CNN for the corresponding task should outperform a method which instead explicitly models part appearances and relations.”
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters (2015, background): “able this and other reinforcement learning applications to move toward real-time. Deep Neural Networks are used for an ever-broadening variety of problems, including classifying [41,24] and detecting [22,17] objects in images, writing sentences about images [15], identifying actions in videos [4], p”
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups (2016, background): “formable parts model (DPM). If we assume that lter responses identify parts (or more elemental features), then there should be more lter co-dependence with depth, as more complex relationships emerge [7]. In general, state-of-the-art CNN architectures have few lters in early convolutional layers, in”
Pose2Seg: Detection Free Human Instance Segmentation (2018, methodology): “Some works [4, 10, 12, 13] employ a multi-stage pipeline which first uses detection to generate bounding boxes and then applies semantic segmentation.”
Deep Learning Strong Parts for Pedestrian Detection (2015, background): “For example, according to the statistics of the occlusion patterns in traffic scenes, [18] trained a series of biased classifiers for bottom-up and right-left occlusions.”
On the relationship between visual attributes and convolutional networks (2015, background): “1The relationship between conv-nets and the deformable parts model (DPM) has been recently studied in [6].”
Higher Order Potentials in End-to-End Trainable Conditional Random Fields (2015, methodology): “Graphical models have been used with CNNs in other areas besides semantic segmentation, such as in poseestimation [41], deformable part models [13], and group activity recognition [7].”