InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

By Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, Kyoung Mu Lee. European Conference on Computer Vision, 2020

402 citations recorded by the source corpus.

Analysis of hand-hand interactions is a crucial step towards better understanding human behavior. However, most researches in 3D hand pose estimation have focused on the isolated single hand case. Therefore, we firstly propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network, InterNet, for 3D interacting hand pose estimation from a single RGB image. The proposed InterHand2.6M consists of \textbf{2.6M labeled single and interacting hand frames} under various poses from multiple subjects. Our InterNet simultaneously performs 3D single and interacting hand pose estimation. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset. Finally, we show 3D interacting hand pose estimation results from general images. Our code and dataset are available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/d93df7ff9453d8ccd3d7d609d0e90b00ec74ae77

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

DexYCB: A Benchmark for Capturing Hand Grasping of Objects (2021, neutral): “6M [22] RGB X X X × – X 512×334 2,590K 27 – >80 X – – semi-auto Dexter+Object [32] RGB-D X X × X X × 640×480 3K 2 2 1 X 6 X manual Simon et al.”
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives (2023, neutral): “Some methods assume no body visibility [63, 80, 98, 187, 188], while others assume partial observability by modifying cameras to capture the body [3, 57, 136, 161, 181].”
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition (2021, methodology): “A few datasets collect hand pose ground truth, acquired in an automated or semi-automated way (Panoptic [37, 38], FreiHand [99], Interhand [55]).”
Towards Unconstrained Joint Hand-Object Reconstruction From RGB Videos (2021, background): “Existing datasets rely on depth data [10, 16], visible marker equipment [14, 52] or multiple views [6, 44, 56, 68].”
Reconstructing Hands in 3D with Transformers (2023, methodology): “For training data, we use multiple available sources of data with hand annotations, including both studio/controlled datasets with 3D ground truth [6, 19, 40, 56, 63, 64], and in-the-wild datasets annotated with 2D keypoint locations [15, 25, 52].”
Effective Whole-body Pose Estimation with Two-stages Distillation (2023, methodology): “Due to the lack of whole-body annotations, most previous models are designed for bodyonly [22, 40, 47, 53], hand-only [6, 32, 62, 68], or faceonly [21, 46, 67].”
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation (2022, methodology): “Following [10], we use an iterative refinement scheme to predict parameters of the hands and the objects.”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, methodology): “adopted, namely 300Wface [61], FreiHand [62] and InterHand [63].”