Building Part-Based Object Detectors via 3D Geometry

By Abhinav Shrivastava, Abhinav Kumar Gupta. 2013

43 citations recorded by the source corpus.

This paper proposes a novel part-based representation for modeling object categories. Our representation combines the effectiveness of deformable part-based models with the richness of geometric representation by defining parts based on consistent underlying 3D geometry. Our key hypothesis is that while the appearance and the arrangement of parts might vary across the instances of object categories, the constituent parts will still have consistent underlying 3D geometry. We propose to learn this geometry-driven deformable part-based model (gDPM) from a set of labeled RGBD images. We also demonstrate how the geometric representation of gDPM can help us leverage depth data during training and constrain the latent model learning problem. But most importantly, a joint geometric and appearance based representation not only allows us to achieve state-of-the-art results on object detection but also allows us to tackle the grand challenge of understanding 3D objects from 2D images.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Single Image 3D Interpreter Network (2016, background): “Depth-/mesh-based models can recover detailed 3D object structure from a single image, either by adapting existing 3D models from a database [3,19,6,8,20,21,22,23,24], or by inferring from its detected 2D silhouette [5,7,4].”
Matterport3D: Learning from RGB-D Data in Indoor Environments (2017, methodology): “These previous RGB-D datasets have been used to train models for several standard scene understanding tasks, including semantic segmentation [27, 33, 13, 40, 7], 3D object detection [31, 24, 14, 36, 37], normal estimation [41, 23, 9, 3], camera relocalization [30, 39], and others [47, 11, 12].”
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene (2017, background): “For instance, researchers have predicted 3D object pose [23, 32], low-dimensional parametric shapes [8, 34], and surface normals [28].”
Is object localization for free? - Weakly-supervised learning with convolutional neural networks (2015, unknown): “The target output in visual recognition ranges from image-level labels (object/image classification) [23], location and extent of objects in the form of bounding boxes (object detection) [18], to detailed object segmentation [5, 58, 24] or even predicting an approximate 3D pose and geometry of objec”
Learning Rich Features from RGB-D Images for Object Detection and Segmentation (2014, result): “While quite low, this result agrees with [32].”
Learning with Side Information through Modality Hallucination (2016, background): “We represent the depth images using the HHA encoding introduced by Gupta et al. [15] and independently finetune the depth network after initializing with the RGB weights.”
SUN RGB-D: A RGB-D scene understanding benchmark suite (2015, background): “There are many interesting works on RGB-D scene understanding , including semantic segmentation [53] [49] [19] object classification [69], object detection [59] [20] [62], context reasoning [38], mid-level recognition [32, 31], and surface orientation and room layout estimation [13, 14, 74].”
Sliding Shapes for 3D Object Detection in Depth Images (2014, methodology): “In RGBD/3D case we compare with [24] which use 2D HOG on RGBD image with 2D sliding window, and gDPM [25] which trains a geometry driven deformable part model.”