Hands in action: real-time 3D reconstruction of hands in interaction with objects

By Javier Romero, Hedvig Kjellström, Danica Kragic. 2010

143 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “One highly related direction is to model and estimate 3D hand joints [5,44,45,53,60,63,66,72,78,79,80,85,3] and mesh [26,27,51,55,75,83,46], object pose [14,28,29,35,43,57,64,71,76,81], or both [6,17,18,19,69].”
Hand Pose Estimation via Latent 2.5D Heatmap Regression (2018, methodology): “These methods follow a non-parametric approach and formulate 3D pose estimation as a nearest neighbor search problem in large databases of 3D poses, where the matching is performed based on some low [38, 39] or high [40,41] level features extracted from the image.”
Motion Capture of Hands in Action Using Discriminative Salient Points (2012, methodology): “In the context of object grasping, [5] built a database of 10, 000 hand poses with and without grasped objects to recover the hand pose from images using nearest neighbor search.”
Reconstructing Hand-Object Interactions in the Wild (2020, background): “Many previous works on hand pose estimation directly predict 3D joint locations from either depth [50, 53, 55, 59, 64, 65, 35] or RGB [45, 36, 4, 63, 68] images.”
Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters (2017, background): “3D Models for Detection: A notable application of 3D computer graphics model in vision has been the modeling of the human body shapes [18, 4, 1, 29, 36, 3, 41].”
Real-Time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor (2017, unknown): “Some methods can track hands even when they interact with objects [6, 27], but they are limited to slow motions and limited articulation.”
GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes (2020, background): “A broad line of work aims to estimate the pose of a hand manipulating an object, either from RGB-D inputs [26, 25, 62, 65, 66], video sequences [57, 4, 48, 68] or single images [34, 35].”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, background): “[40] obtain 3D hand-object reconstructions via nearest neighbor search in a large database of synthetic images.”
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild (2016, methodology): “Synthetic data have been used for upper body [29], full-body silhouettes [1], hand-object interactions [28], full-body pose from depth [30] or egocentric RGB-D scenes [27].”
Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data (2013, background): “Other generative approaches have been proposed that use objects being manipulated by the hand as constraints [7, 8, 15, 19].”
A Survey on Human Motion Analysis from Depth Data (2013, methodology): “Although these methods process the frames independently, temporal consistency can be enforced [135, 136, 137].”
Learning scene-specific pedestrian detectors without real data (2015, methodology): “3D computer graphics models have been used for modeling human shape [16, 7], body-part gradient templates [1], full-body gradient templates [22] and hand appearance [27, 2, 28].”
GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB (2017, methodology): “Some of the first methods for this problem [12, 43, 30] did not produce metrically accurate 3D pose as they only fetched the nearest 3D”
Full DOF tracking of a hand interacting with an object by modeling occlusions and physical constraints (2011, methodology): “Closely related to our problem, a few re cent works [12, 8, 29, 21] consider context for classifying human-object interaction activities.”
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (2019, background): “While reasoning about hands in action, object interactions can be exploited as additional constraints [41, 49, 51, 53].”
3D Hand Shape and Pose From Images in the Wild (2019, background): “When it comes to a monocular color input, the problem becomes inherently ill posed due to the increased depth and scale ambiguities, but that did not prevent several researchers [4, 9, 51, 57, 63, 39] from attempting to solve it in the past albeit with limited results.”
GRAB: A Dataset of Whole-Body Human Grasping of Objects (2020, methodology): “contact and physics, for body-scene [6, 22, 23, 57, 76], body-object [30, 32, 37], or hand-object [24, 45, 54, 55, 62, 68, 69, 72, 77, 78] scenarios.”