PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation

By Yisheng He, Wei Sun, Haibin Huang, Jianran Liu, Haoqiang Fan, Jian‐Yuan Sun. Computer Vision and Pattern Recognition, 2019

593 citations recorded by the source corpus.

In this work, we present a novel data-driven method for robust 6DoF object pose estimation from a single RGBD image. Unlike previous methods that directly regressing pose parameters, we tackle this challenging task with a keypoint-based approach. Specifically, we propose a deep Hough voting network to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner. Our method is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It allows us to fully utilize the geometric constraint of rigid objects with the extra depth information and is easy for a network to learn and optimize. Extensive experiments were conducted to demonstrate the effectiveness of 3D-keypoint detection in the 6D pose estimation task. Experimental results also show our method outperforms the state-of-the-art methods by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/PVN3D.git.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/7d83345b0ac59e583a14f73e65a1715ed462e76a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Bowen Wen
2. Stanley T. Birchfield
3. Jan Kautz
4. Wei Yang

In Computer Vision and Pattern Recognition

1. Xiaolong Wang
2. Yang Fu
3. Bowen Wen
4. Yuzhe Qin
5. Ruihan Yang
6. Shaowei Liu
7. Hanwen Jiang
8. Stanley T. Birchfield

In Control and Systems Engineering

1. Bowen Wen
2. Xiaolong Wang
3. Yuzhe Qin
4. Ruihan Yang
5. Shaowei Liu
6. Hanwen Jiang
7. Stanley T. Birchfield
8. Jan Kautz

Citations in the corpus

• GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, unknown): “Nonetheless, with the advent of deep learning, meth-odsbasedonneuralnetworksbecomedominantininstance-level objectposeestimation[16],[17],[18],[19],[20],[21].”
• FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, methodology): “Instance-level pose estimation methods [19, 20, 30, 47] assume a textured CAD model is given for the object.”
• DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, methodology): “We estimate the object pose using a single camera from the front view, which matches the object views where the PVN3D model is trained on.”