D3D-HOI: Dynamic 3D Human-Object Interactions from Videos

By Hanbyul Joo, Greg Mori, Manolis Savva. 2021

49 citations recorded by the source corpus.

We introduce D3D-HOI: a dataset of monocular videos with ground truth annotations of 3D object pose, shape and part motion during human-object interactions. Our dataset consists of several common articulated objects captured from diverse real-world scenes and camera viewpoints. Each manipulated object (e.g., microwave oven) is represented with a matching 3D parametric model. This data allows us to evaluate the reconstruction quality of articulated objects and establish a benchmark for this challenging task. In particular, we leverage the estimated 3D human pose for more accurate inference of the object spatial layout and dynamics. We evaluate this approach on our dataset, demonstrating that human-object relations can significantly reduce the ambiguity of articulated object reconstructions from challenging real-world videos. Code and dataset are available at https://github.com/facebookresearch/d3d-hoi.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

In Human-Computer Interaction

Citations in the corpus

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, background): “VISOR is also related to efforts that capture hand-object relations, including for identifying active objects [35, 4], next active objects [33, 25], object articulation [44] and grasping [18, 39, 38, 40].”