Segmentation-Driven 6D Object Pose Estimation

By Yinlin Hu, Joachim Hugonot, Pascal Fua, Mathieu Salzmann. Computer Vision and Pattern Recognition, 2018

318 citations recorded by the source corpus.

The most recent trend in estimating the 6D pose of rigid objects has been to train deep networks to either directly regress the pose from the image or to predict the 2D locations of 3D keypoints, from which the pose can be obtained using a PnP algorithm. In both cases, the object is treated as a global entity, and a single pose estimate is computed. As a consequence, the resulting techniques can be vulnerable to large occlusions. In this paper, we introduce a segmentation-driven 6D pose estimation framework where each visible part of the objects contributes a local pose prediction in the form of 2D keypoint locations. We then use a predicted measure of confidence to combine these pose candidates into a robust set of 3D-to-2D correspondences, from which a reliable pose estimate can be obtained. We outperform the state-of-the-art on the challenging Occluded-LINEMOD and YCB-Video datasets, which is evidence that our approach deals well with multiple poorly-textured objects occluding each other. Furthermore, it relies on a simple enough architecture to achieve real-time performance.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/b7e5d2c78d0eb9cd6db2c1365697ecc08318b94c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, background): “One line of work focus on object pose estimation [36, 56, 85, 79, 50, 30, 27] and hand pose estimation [87, 31, 76, 23, 5, 11, 26, 38, 42].”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, methodology): “[Hu et al., 2019], DPOD [Zakharov et al.”
iGibson, a Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, background): “as: depth/segmentation/normal/affordance prediction [45, 46, 47], action-conditioned flow prediction [48], multi-modal pose estimation [49, 50, 51], and visuomotor policy training assuming perfect vision systems [35, 52].”
GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes (2020, methodology): “1) For multi-object scenes, we assume the observed objects are known and integrate the state-of-the-art object pose estimation method [29] to estimate their 6D pose, which we denote as Tobject.”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, background): “Alternative keypoint representations include vector-fields [34] and patches [14].”
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, background): “These modalities can support research topics such as: depth/segmentation/normal/affordance prediction [48, 49, 50], action-conditioned flow prediction [51], multi-modal pose estimation [52, 53, 54], and visuomotor policy training assuming perfect vision systems [35, 55].”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “[41, 47, 20] directly regress the 2D coordinate of the keypoints, while [33, 24, 34] use heatmaps to locate the 2D keypoints.”
HOnnotate: A Method for 3D Annotation of Hand and Object Poses (2019, methodology): “Some methods are now robust to partial occlusions [20, 36, 41], but many works rely on RGB-D data to handle this problem [5, 8, 22, 30], by fitting the 3D object model to depth data.”