Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
By Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov, Kun He, Dipika Singhania, Robert Wang, Angela Yao. Computer Vision and Pattern Recognition, 2022
379 citations recorded by the source corpus.
Assembly101 is a new procedural activity dataset fea-turing 4321 videos of people assembling and disassembling 101 “take-apart” toy vehicles. Participants work without fixed instructions, and the sequences feature rich and natu-ral variations in action ordering, mistakes, and corrections. Assembly101 is the first multi-view action dataset, with si-multaneous static (8) and egocentric (4) recordings. Se-quences are annotated with more than 100K coarse and 1M fine-grained action segments, and I8M 3D hand poses. We benchmark on three action understanding tasks: recognition, anticipation and temporal segmentation. Ad-ditionally, we propose a novel task of detecting mistakes. The unique recording format and rich set of annotations al-low us to investigate generalization to new toys, cross-view transfer, long-tailed distributions, and pose vs. appearance. We envision that Assemblyl0l will serve as a new challenge to investigate various activity understanding problems.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing
View Paper: https://www.semanticscholar.org/paper/8699794561b74e461fa86e1a9dcd5de74d6d7f6d
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Giovanni Maria Farinella
- Antonino Furnari
- Francesco Ragusa
- Ilija Radosavovic
- Jitendra Malik
- Angjoo Kanazawa
- David F. Fouhey
- Dandan Shan
In Computer Vision and Pattern Recognition
- Jitendra Malik
- Kristen Grauman
- Luc Van Gool
- Cordelia Schmid
- Trevor Darrell
- Andrew Zisserman
- Giovanni Maria Farinella
- Antonino Furnari
In Signal Processing
- Giovanni Maria Farinella
- Antonino Furnari
- Jitendra Malik
- Angjoo Kanazawa
- Francesco Ragusa
- Ilija Radosavovic
Citations in the corpus
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives (2023, neutral): “More recent ego-exo efforts focus on specific activities in one or two environments.”
- MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain (2022, methodology): “Assembly101 [81] Industrial-like X X RGB, multi-view, 3D hand-pose AR, AA, TAS, MS 2022 20M 362 7.”
- Reconstructing Hands in 3D with Transformers (2023, background): “AssemblyHands [42] annotated 3D hand poses for synchronized images from Assembly101 [50] which participants assemble and disassemble take-apart toys in a multi-camera setting.”