Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

By Kiru Park, T. Patten, Markus Vincze. IEEE International Conference on Computer Vision, 2019

538 citations recorded by the source corpus.

Estimating the 6D pose of objects using only RGB images remains challenging because of problems such as occlusion and symmetries. It is also difficult to construct 3D models with precise texture without expert knowledge or specialized scanning devices. To address these problems, we propose a novel pose estimation method, Pix2Pose, that predicts the 3D coordinates of each object pixel without textured models. An auto-encoder architecture is designed to estimate the 3D coordinates and expected errors per pixel. These pixel-wise predictions are then used in multiple stages to form 2D-3D correspondences to directly compute poses with the PnP algorithm with RANSAC iterations. Our method is robust to occlusion by leveraging recent achievements in generative adversarial training to precisely recover occluded parts. Furthermore, a novel loss function, the transformer loss, is proposed to handle symmetric objects by guiding predictions to the closest symmetric pose. Evaluations on three different benchmark datasets containing symmetric and occluded objects show our method outperforms the state of the art using only RGB images.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/527c39c22ee1f9ba91eec359185c1ebcb4cd909a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “that operate on RGB (no depth) images [7,8,10–12] are based on trainable convolutional neural networks and are able to deal with symmetric or textureless objects, which were challenging for earlier methods relying on local [3,13–16] or global [17] gradient-based image features.”
BOP Challenge 2020 on 6D Object Localization (2020, methodology): “ 23.7 48.7 61.4 28.1 15.8 50.6 50.5 0.86 20 Zhigang-CDPN-ICCV19 [35] 35.3 37.4 12.4 75.7 25.7 7.0 47.0 42.2 0.51 21 PointVoteNet2 [15] 35.1 65.3 0.4 67.3 26.4 0.1 55.6 30.8 - 22 Pix2Pose-BOP20-ICCV19 [40] 34.2 36.3 34.4 42.0 22.6 13.4 44.6 45.7 1.22 23 Sundermeyer-IJCV19 [49] 27.0 14.6 30.4 40.1 21.”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, unknown): “Recently, Pix2Pose [30] improves robustness against occlusions by predicting dense 2D-3D correspondences between image pixels and the object model.”
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, methodology): “Instance-level pose estimation methods [19, 20, 30, 47] assume a textured CAD model is given for the object.”
DexYCB: A Benchmark for Capturing Hand Grasping of Objects (2021, background): “State-of-the-art approaches for both 3D object pose [37, 20, 34, 27, 40, 26, 19] and 3D hand pose estimation [49, 23, 18, 2, 9, 14, 31] rely on deep learning and thus require large datasets with labeled hand or object poses for training.”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, methodology): “[38], BB8 [36], Pix2Pose [30], PVNet [34], CDPN [20], and DPOD [44].”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, unknown): “Compared with indirect methods which rely on 2D-3D or 3D-3D correspondence like [24], [45], our method offers a compelling combination of real-time performance and accurate pose estimation.”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “They either use random forest [3, 32] or CNNs [23, 9, 27, 35, 51] to extract feature and predict the corresponding 3D object coordinates for each pixel and then vote for the final pose results.”