Inferring 3D Object Pose in RGB-D Images

By Saurabh Gupta, Pablo Arbeláez, Ross B. Girshick, Jitendra Malik. 2015

33 citations recorded by the source corpus.

The goal of this work is to replace objects in an RGB-D scene with corresponding 3D models from a library. We approach this problem by first detecting and segmenting object instances in the scene using the approach from Gupta et al. [13]. We use a convolutional neural network (CNN) to predict the pose of the object. This CNN is trained using pixel normals in images containing rendered synthetic objects. When tested on real data, it outperforms alternative algorithms trained on real data. We then use this coarse pose estimate along with the inferred pixel support to align a small number of prototypical models to the data, and place the model that fits the best into the scene. We observe a 48% relative improvement in performance at the task of 3D detection over the current state-of-the-art [33], while being an order of magnitude faster at the same time.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning a Predictable and Generative Vector Representation for Objects (2016, methodology): “Among a large body of works trying to infer 3D representations from images, our approach is most related to a group of works using renderings of 3D CAD models to predict properties such as object viewpoint [33] or class [32], among others [10, 25, 31].”
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views (2015, methodology): “Recently, [32, 19, 13, 27] used 3D models to render images for training object detectors and viewpoint classifiers.”
Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories (2016, background): “While benchmark scores for many CNN intrinsics models [21, 34, 35, 20, 22] are improving, the visual quality of results remains poor, compared with those from traditional approaches based on hand-crafted features and multitudes of priors [6].”
Synthetic Data for Deep Learning (2019, background): “[225] and multi-view object class detection in Liebelt and Schmid [359] and Stark et al.”