The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain

By Francesco Ragusa, Antonino Furnari, Salvatore Livatino, Giovanni Maria Farinella. 2020

138 citations recorded by the source corpus.

Wearable cameras allow to collect images and videos of humans interacting with the world. While human-object interactions have been thoroughly investigated in third person vision, the problem has been understudied in egocentric settings and in industrial scenarios. To fill this gap, we introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. MECCANO has been acquired by 20 participants who were asked to build a motorbike model, for which they had to interact with tiny objects and tools. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Specifically, each interaction has been labeled both temporally (with action segments) and spatially (with active object bounding boxes). With the proposed dataset, we investigate four different tasks including 1) action recognition, 2) active object detection, 3) active object recognition and 4) egocentric human-object interaction detection, which is a revisited version of the standard human-object interaction detection task. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios. We publicy release the dataset at https://iplab.dmi.unict.it/MECCANO/.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, background): “3 [34] Francesco Ragusa, Antonino Furnari, Salvatore Livatino, and Giovanni Maria Farinella.”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “Aside from daily life activity, prior ego datasets focus on conversation [170], inter-person interactions [66, 168, 194, 231], place localization [183, 208], multimodal sensor data [124, 166, 204], human hands [16, 134] human-object interaction [106, 184], and object tracking [56].”
Is First Person Vision Challenging for Object Tracking? (2020, unknown): “Understanding the interactions between a camera wearer and the surrounding objects is a fundamental problem in First Person Vision (FPV) [19, 81, 51, 30, 20, 68, 12, 5, 6].”
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain (2022, methodology): “Table 1 compares the aforementioned datasets with respect to the proposed MECCANO Multimodal dataset which is a conspicuous extension of the previous MECCANO dataset [74].”
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives (2023, unknown): “[10, 126, 145, 148] or third-person [9, 100, 159, 205, 207] videos; however, limited work considers both views together.”
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition (2021, background): “While many datasets for thirdperson action recognition have been proposed throughout the years [30, 40, 69, 96], recently there is a surge in interest for data targeting also egocentric scenarios [12, 13, 29, 47, 60, 68] that involve mostly 2D features.”
Predicting the Future from First Person (Egocentric) Vision: A Survey (2021, background): “MECCANO [24]: Dataset of egocentric videos human-object interactions in industrial-like settings.”