TutorialVQA: Question Answering Dataset for Tutorial Videos

By Anthony Colas, Seokhwan Kim, Franck Dernoncourt, Siddhesh Gupte, Daisy Zhe Wang, Doo Soon Kim. 2019

36 citations recorded by the source corpus.

Despite the number of currently available datasets on video-question answering, there still remains a need for a dataset involving multi-step and non-factoid answers. Moreover, relying on video transcripts remains an under-explored topic. To adequately address this, we propose a new question answering task on instructional videos, because of their verbose and narrative nature. While previous studies on video question answering have focused on generating a short text as an answer, given a question and video clip, our task aims to identify a span of a video segment as an answer which contains instructional details with various granularities. This work focuses on screencast tutorial videos pertaining to an image editing program. We introduce a dataset, TutorialVQA, consisting of about 6,000 manually collected triples of (video, question, answer span). We also provide experimental results with several baseline algorithms using the video transcripts. The results indicate that the task is challenging and call for the investigation of new algorithms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Question Answering & Reading Comprehension

Citations in the corpus

• Just Ask: Learning to Answer Questions from Millions of Narrated Videos (2020, neutral): “As result, current VideoQA datasets [12, 17, 18, 22, 28, 35, 40, 45, 48, 62, 71, 77, 87, 93, 94, 95, 96] are limited in size, as the largest, TGIF-QA [35], contains only 72K annotated clips (see Figure 2 for more details).”
• Zero-Shot Video Question Answering via Frozen Bidirectional Language Models (2022, unknown): “[17] Anthony Colas, Seokhwan Kim, Franck Dernoncourt, Siddhesh Gupte, Daisy Zhe Wang, and Doo Soon Kim.”