EgoVQA - An Egocentric Video Question Answering Benchmark Dataset

By Chenyou Fan. 2019

79 citations recorded by the source corpus.

Recently, much effort and attention has been devoted to Visual Question Answering (VQA) on static images and Video Question Answering (VideoQA) on third-person videos. In the meantime, first-person question answering has more natural use cases while this topic remains seldom studied. A typical meaningful scenario is an intelligent agent provides assistance to handicapped people to perceive the environment by the queries, localize objects and persons based on descriptions, and identify intentions of surrounding people to guide their reactions (e.g., shake hands or avoid punches). However, due to the lack of first-person video datasets, seldom study had been carried on first-person VideoQA task. To address this issue, we collected a novel egocentric VideoQA dataset called EgoVQA with 600 question-answer pairs with visual contents across 5,000 frames from 16 first-person videos. Various types of queries such as "Who", "What", "How many" are provided to form a semantically rich corpus. We use this database to evaluate the performance of four mainstream third-person VideoQA methods to illustrate their performance gap between first-person related questions and third-person related questions. We believe that EgoVQA dataset will facilitate future research on the imperative task of first-person VideoQA.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Question Answering & Reading Comprehension

Citations in the corpus

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (2021, methodology): “We use the implementation provided by [8] as there is no official code available for CoMem.”
Just Ask: Learning to Answer Questions from Millions of Narrated Videos (2020, neutral): “As result, current VideoQA datasets [12, 17, 18, 22, 28, 35, 40, 45, 48, 62, 71, 77, 87, 93, 94, 95, 96] are limited in size, as the largest, TGIF-QA [35], contains only 72K annotated clips (see Figure 2 for more details).”
Data augmentation techniques for the Video Question Answering task (2020, methodology): “ VideoQA nor for the VideoQA task, in this work we propose several data augmentation techniques which exploit characteristics given by the task itself. In particular, by exploiting the EgoVQA dataset [6] we show their impact on the nal performance obtained by the STVQA model [12],which is proven to ”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “EgoVQA [227] shifts the focus of VQA to egocentric videos.”
On Modality Bias in the TVQA Dataset (2020, unknown): “Multiple video question answering (video-QA) datasets have been developed (MovieQA [42], MovieFIB [36], PororoQA [29], TGIF-QA [26], YouTube2Text-QA [47], EgoVQA [15] and TVQA [32]).”
Recent Advances in Video Question Answering: A Review of Datasets and Methods (2021, methodology): “EgoVQA [34] 520 600 Manual Egocentric video studies Open-ended and multiple-choice”
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs (2024, unknown): “…of publicly available datasets, including Kinetics-710 (Kay et al., 2017), SthSthv2 (Goyal et al., 2017), NExtQA (Xiao et al., 2021), CLEVRER (Yi et al., 2019), EgoQA (Fan, 2019), Tgif (Li et al., 2016), WebVidQA(Yang et al., 2021), RealworldQA (X. AI, 2024), and Hm3d (Ramakrishnan et al., 2021).”
LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding (2024, unknown): “…(Wu et al., 2025), Youcook2 (Zhou et al., 2018), Kinetics-710 (Kay et al., 2017), NExTQA (Xiao et al., 2021), CLEVRER (Yi et al., 2019), EgoQA (Fan, 2019), TGIF (Li et al., 2016), WebVidQA (Yang et al., 2021), ShareGPT4Video (Chen et al., 2024), and MovieChat (Song et al., 2024) as the long…”