Real-Time Seamless Single Shot 6D Object Pose Prediction

By Bugra Tekin, Sudipta N. Sinha, Pascal Fua. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

894 citations recorded by the source corpus.

We propose a single-shot approach for simultaneously detecting an object in an RGB image and predicting its 6D pose without requiring multiple stages or having to examine multiple hypotheses. Unlike a recently proposed single-shot technique for this task [10] that only predicts an approximate 6D pose that must then be refined, ours is accurate enough not to require additional post-processing. As a result, it is much faster - 50 fps on a Titan X (Pascal) GPU - and more suitable for real-time processing. The key component of our method is a new CNN architecture inspired by [27, 28] that directly predicts the 2D image locations of the projected vertices of the object's 3D bounding box. The object's 6D pose is then estimated using a PnP algorithm. For single object and multiple object pose estimation on the LINEMOD and OCCLUSION datasets, our approach substantially outperforms other recent CNN-based approaches [10, 25] when they are all used without postprocessing. During post-processing, a pose refinement step can be used to boost the accuracy of these two methods, but at 10 fps or less, they are much slower than our method.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/646a669f1dc38ae961fe41fbd3c83cab64ce9d53

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects (2018, methodology): “While deep neural networks have been successfully applied to the problem of object detection in 2D [1, 2, 3], they have only recently begun to be applied to 3D object detection and pose estimation [4, 5, 6].”
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition (2021, background): “While a significant amount of research has focused on predicting the pose of hands [27, 54, 56, 58, 70, 93, 94, 98] or objects [5, 48, 59, 78, 83, 90] in isolation, joint understanding of handobject interactions has received far less attention.”
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation (2019, methodology): “4 Tekin [30] 21.”
iNeRF: Inverting Neural Radiance Fields for Pose Estimation (2020, methodology): “Recent approaches based on deep learning have proposed to 1) directly estimate objects pose using CNNbased architectures [32], [40], [46] or 2) estimate 2D keypoints [27], [35], [37], [38] and solve for pose using the PnP-RANSAC algorithm.”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “[41, 47, 20] directly regress the 2D coordinate of the keypoints, while [33, 24, 34] use heatmaps to locate the 2D keypoints.”
DexYCB: A Benchmark for Capturing Hand Grasping of Objects (2021, background): “State-of-the-art approaches for both 3D object pose [37, 20, 34, 27, 40, 26, 19] and 3D hand pose estimation [49, 23, 18, 2, 9, 14, 31] rely on deep learning and thus require large datasets with labeled hand or object poses for training.”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “by rendering the corresponding 3D model. Recently, 2D object detection methods are used as template matching and augmented for 6D pose estimation, especially with deep learning-based object detectors [28, 23, 16, 29]. Templatebased methods are useful in detecting texture-less objects. However, they ”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “A convolutional neural network (CNN) can be used to detect object features in 2D [4,6,18,21,22] or to directly find 2D-to-3D correspondences [5,7,8,23].”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, methodology): “Some recent methods [29, 30, 36] use CNNs to first regress 2D keypoints and then compute 6D pose parameters using the Perspective-n-Point (PnP) algorithm.”
Segmentation-Driven 6D Object Pose Estimation (2018, methodology): “Two main trends have emerged: Either regressing from the image directly to the 6D pose [17, 45] or predicting 2D keypoint locations in the image [35, 39], from which the pose can be obtained via PnP.”
CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation (2019, methodology): “Here we compare with others on 2D Projection for some approaches ([25], [20]) only report results on this metric.”
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape (2018, background): “Instead of discretizing SO(3), [34, 37] formulate the 6D estimation problem as a regression of the 2D projections of the 3D bounding box.”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, neutral): “As in previous work [21, 47, 48], we assume that a 3D mesh model of the object is provided.”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, background): “[38], BB8 [36], Pix2Pose [30], PVNet [34], CDPN [20], and DPOD [44].”
Imbalance Problems in Object Detection: A Review (2019, background): “surveillance [1], [2], autonomous driving [3], [4], medical decision making [5], [6], and many problems in robotics [7], [8], [9], [10], [11], [12].”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, neutral): “For instance, BB8 [37] and YOLO6D [38] compute the 2D projections of a set of ﬁxed control points (e.g.the 3D corners of the encapsulating bounding box).”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “Deep-learning methods for viewpoint estimation follow roughly three different paths: direct estimation of Euler angles [55, 50, 33, 24, 62, 63], template-based matching [20, 32, 51], and keypoint detection relying on 3D bounding box corners [39, 52, 13, 34, 36] or semantic keypoints [35, 65].”
Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation (2020, background): “pose from RGB-D images in an end-to-end framework. The third category relies on establishing point correspondences. [2,12,17] regress the corresponding 3D object coordinate for each foreground pixel. [18,23,28] detect the keypoints of the object on image and then solve a Perspective-n-Point problem.”