Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
By Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote and 91 more. Computer Vision and Pattern Recognition, 2023
529 citations recorded by the source corpus.
We present Ego-Exo4D, a diverse, large-scale multi-modal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured ego-centric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair). 740 participants from 13 cities worldwide performed these activities in 123 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,286 hours of video combined. The multimodal nature of the dataset is un-precedented: the video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU, and multiple paired language descriptions-including a novel “expert commentary” done by coaches and teachers and tailored to the skilled-activity domain. To push the frontier of first-person video understanding of skilled human activity, we also present a suite of benchmark tasks and their annotations, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose. All resources are open sourced to fuel new research in the community.
Subfields: Applied Psychology, Clinical Psychology, Developmental and Educational Psychology
View Paper: https://www.semanticscholar.org/paper/2f053d26f76df3327651b65e14d4ea24aa8772fa
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Christoph Feichtenhofer
- Junting Pan
- Wenguan Wang
- Jianwei Yang
- Rohit Girdhar
- Kaiming He
- Kristen Grauman
- Luc Van Gool
Citations in the corpus
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (2025, unknown): “While image pairs are an important use case, often one is interested in re-constructing from more than two views, as when scanning of objects [39] or scenes [4, 6, 20, 55, 67], e.g . for asset generation or mapping.”
- SAM 2: Segment Anything in Images and Videos (2024, background): “…et al., 2017), VIPSeg (Miao et al., 2022), Drosophila Heart OCM (DH OCM) (Fishman et al., 2023), EndoVis 2018 (Allan et al., 2020), ESD (Huang et al., 2023), UVO (Wang et al., 2021b), Ego-Exo4d (Grauman et al., 2023), LVOSv2 (Hong et al., 2024), and HT1080WT (Gómez-de Mariscal et al., 2021).”
- AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning (2025, unknown): “…and Virtanen 2025], multi-modal large language models [Chowdhury et al. 2025c, 2024a, 2025d, Sun et al. 2024a, Zhang, Li, and Bing 2023b, Zhu et al. 2024, Pramanick et al. 2023a, Lin et al. 2023a], and cross-modal integration [Li et al. 2021, Ghosh et al. 2024, Nag, Goswami, and Karanam 2024,…”