Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals

By Shanxin Yuan, Guillermo Garcia-Hernando, Bjorn Stenger, Gyeongsik Moon, J. Chang, Kyoung Mu Lee, Pavlo Molchanov, Jan Kautz, Sina Honari, Liuhao Ge and 14 more. 2017

46 citations recorded by the source corpus.

In this paper, we strive to answer two questions: What is the current state of 3D hand pose estimation from depth images? And, what are the next challenges that need to be tackled? Following the successful Hands In the Million Challenge (HIM2017), we investigate the top 10 state-of-the-art methods on three tasks: single frame 3D pose estimation, 3D hand tracking, and hand pose estimation during object interaction. We analyze the performance of different CNN structures with regard to hand shape, joint visibility, view point and articulation distributions. Our findings include: (1) isolated 3D hand pose estimation achieves low mean errors (10 mm) in the view point range of [70, 120] degrees, but it is far from being solved for extreme view points; (2) 3D volumetric representations outperform 2D CNNs, better capturing the spatial structure of the depth data; (3) Discriminative methods still generalize poorly to unseen hand shapes; (4) While joint occlusions pose a challenge for most methods, explicit modeling of structure constraints can significantly narrow the gap between errors on visible and occluded joints.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jitendra Malik
2. Grégory Rogez
3. Cordelia Schmid
4. Luc Van Gool
5. Andrew Zisserman
6. Francesc Moreno-Noguer
7. Bernt Schiele
8. Dacheng Tao

In Control and Systems Engineering

1. Francesc Moreno-Noguer
2. Grégory Rogez
3. Enric Corona
4. Guillem Alenyà
5. Albert Pumarola

In 3D Vision & Pose

1. Lingzhi Zhang
2. Francesc Moreno-Noguer
3. Shenghao Zhou
4. Simon Stent
5. Grégory Rogez
6. Guillem Alenyà
7. Jianbo Shi
8. Albert Pumarola

In Human-Computer Interaction

1. Lingzhi Zhang
2. Francesc Moreno-Noguer
3. Shenghao Zhou
4. Simon Stent
5. Grégory Rogez
6. Jianbo Shi

Citations in the corpus

• Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “One highly related direction is to model and estimate 3D hand joints [5,44,45,53,60,63,66,72,78,79,80,85,3] and mesh [26,27,51,55,75,83,46], object pose [14,28,29,35,43,57,64,71,76,81], or both [6,17,18,19,69].”
• GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes (2020, background): “This problem has been studied for years, either taking as input RGB-D [63, 13, 47, 70, 26, 32] or RGB images [71, 49, 46, 61, 31, 2, 5, 21].”