Hands Deep in Deep Learning for Hand Pose Estimation

By Markus Oberweger, Paul Wohlhart, Vincent Lepetit. 2015

385 citations recorded by the source corpus.

We introduce and evaluate several architectures for Convolutional Neural Networks to predict the 3D joint locations of a hand given a depth map. We first show that a prior on the 3D pose can be easily introduced and significantly improves the accuracy and reliability of the predictions. We also show how to use context efficiently to deal with ambiguities between fingers. These two contributions allow us to significantly outperform the state-of-the-art on several challenging benchmarks, both in terms of accuracy and computation times.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

DISCO Nets : DISsimilarity COefficients Networks (2016, methodology): “[17] a constrained prior hand model, referred as NYU-Prior, is refined on each hand joint position to increase accuracy, referred as NYU-Prior-Refined.”
Recurrent Human Pose Estimation (2016, background): “In this respect it is similar to the architecture of [21] which also switches between direct pose co-ordinates and an image representation in an iterated module.”
Deep Kinematic Pose Regression (2016, methodology): “The points could further be the physical joints that deﬁnes the geometry of complex articulated objects, such as human hand [41,21] and human body [17,40,31].”
Structured Prediction of 3D Human Pose with Deep Neural Networks (2016, background): “In the context of hand pose estimation, [21] introduces a bottleneck, lowdimensional layer that aims at accounting for joint dependencies.”
Articulation-Aware Canonical Surface Mapping (2020, methodology): “Perhaps more directly related to our representation is the line of work that, following the seminal work on Blanz and Vetter [3], represents the 3D in the form of a morphabble model, jointly capturing articulation and deformation [21, 25, 26].”
DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System (2019, background): “Impressive strides have also been made in the past few years in bare human hand tracking particularly with deep learning [9, 10, 11, 16, 17, 18].”
Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation (2016, methodology): “We compare our work with 4 state-of-the-arts methods: Hierarchical Sampling Optimization(HSO) [19], Sharp et al. [3], HandsDeep [18], FeedLoop [4] on three datasets, see Fig.”
Learning Joint Reconstruction of Hands and Manipulated Objects (2019, neutral): “The availability of commodity RGB-D sensors [21, 43, 54] led to significant progress in estimating 3D hand pose given depth or RGB-D input [13, 20, 34, 35].”
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals (2017, methodology): “Structured methods embed physical hand motion constraints into the model [11, 20, 21, 29, 55, 61].”
Training a Feedback Loop for Hand Pose Estimation (2015, methodology): “When we use a more complex initialization [21] with an error of 23 mm, we can decrease the average error to 16 mm.”
Disentangling Latent Hands for Image Synthesis and Pose Estimation (2018, background): “As a result, the majority of state-of-the-art methods apply deep learning methods to depth images [5, 6, 7, 8, 14, 18, 19, 32, 33].”
A Review on Deep Learning Techniques Applied to Semantic Segmentation (2017, background): “Some of those applications include autonomous driving [1] [2] [3], human-machine interaction [4], computational photography [5], image search engines [6], and augmented reality to name a few.”
Hand Pose Estimation through Weakly-Supervised Learning of a Rich Intermediate Representation (2015, methodology): “In [14] a deep learning framework is regularized by a bottleneck layer, enforcing the network to model underlying structure of joint positions.”
Monocular Real-Time Hand Shape and Motion Capture Using Multi-Modal Data (2020, background): “Follow-up works achieved better performance by utilizing priors and context [25], high-level knowledge [39], a feedback loop [26, 27], or intermediate dense guidance map supervision [46].”
Cross-Modal Deep Variational Hand Pose Estimation (2018, background): “Furthermore, we explore the utility of the same architecture in the case of depth images and show that we are comparable to state-of-art depth based methods [15, 16, 34] that employ specialized architectures.”
Learning to Estimate 3D Hand Pose from Single RGB Images (2017, methodology): “[12] explored the possibility to encode correlations between keypoint coordinates in a compressing bottleneck, Zhou et al.”