ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

By Samarth Brahmbhatt, Chengcheng Tang, Christopher D. Twigg, Charles C. Kemp, James Hays. 2020

293 citations recorded by the source corpus.

Grasping is natural for humans. However, it involves complex hand configurations and soft tissue deformation that can result in complicated regions of contact between the hand and the object. Understanding and modeling this contact can potentially improve hand models, AR/VR experiences, and robotic grasping. Yet, we currently lack datasets of hand-object contact paired with other data modalities, which is crucial for developing and evaluating contact modeling techniques. We introduce ContactPose, the first dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images. Analysis of ContactPose data reveals interesting relationships between hand pose and contact. We use this data to rigorously evaluate various data representations, heuristics from the literature, and learning methods for contact modeling. Data, code, and trained models are available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose, Human-Computer Interaction, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Jitendra Malik
3. Cordelia Schmid
4. Bernt Schiele
5. Xiaogang Wang
6. Andrew Zisserman
7. Ming-Hsuan Yang
8. Trevor Darrell

In Control and Systems Engineering

1. Michael J. Black
2. Cordelia Schmid
3. Dimitrios Tzionas
4. Gul Varol
5. Ivan Laptev
6. Omid Taheri
7. Xiaolong Wang
8. Shaowei Liu

In 3D Vision & Pose

1. Michael J. Black
2. Cordelia Schmid
3. Ivan Laptev
4. Gul Varol
5. Omid Taheri
6. Muhammed Kocabas
7. Dimitrios Tzionas
8. Yana Hasson

In Human-Computer Interaction

1. Michael J. Black
2. Gul Varol
3. Omid Taheri
4. Cordelia Schmid
5. Muhammed Kocabas
6. Dimitrios Tzionas
7. Ivan Laptev
8. Lingzhi Zhang

In Robotics & Control

1. Xiaolong Wang
2. Shaowei Liu
3. Hanwen Jiang
4. Michael J. Black
5. Jiashun Wang
6. Lingzhi Zhang
7. Sven J. Dickinson
8. Dylan Turpin

Citations in the corpus

• Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “One highly related direction is to model and estimate 3D hand joints [5,44,45,53,60,63,66,72,78,79,80,85,3] and mesh [26,27,51,55,75,83,46], object pose [14,28,29,35,43,57,64,71,76,81], or both [6,17,18,19,69].”
• Hand-Object Contact Consistency Reasoning for Human Grasps Generation (2021, methodology): “Instead of using the tactile sensors on hand, Brahmbhatt et al. [2, 4] proposed to use thermal cameras to capture object contact maps, which reflects the object common contact regions after grasping.”
• Towards Unconstrained Joint Hand-Object Reconstruction From RGB Videos (2021, methodology): “[6] collects a dataset for 3D printed object models in a restricted capture setup.”
• Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Hands (2022, methodology): “[21, 39, 46, 49, 56, 57] all use analytically synthesized datasets from the GraspIt! simulator [63], whereas [7, 9, 81] use datasets of captured”
• ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation (2022, methodology): “2, 7, 16 [6] Samarth Brahmbhatt, Chengcheng Tang, Christopher D.”
• Grasping Field: Learning Implicit Representations for Human Grasps (2020, background): “Concurrent to this work, [7] proposed a new dataset of hand-object contact paired with RGB-D images.”
• DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, background): “Another line of datasets focus on hand-object contact reasoning [12, 17, 78, 14].”