TVQA: Localized, Compositional Video Question Answering

By Jie Lei, Licheng Yu, Mohit Bansal, Tamara L. Berg. 2018

791 citations recorded by the source corpus.

Recent years have witnessed an increasing interest in image-based question-answering (QA) tasks. However, due to data limitations, there has been much less work on video-based QA. In this paper, we present TVQA, a large-scale video QA dataset based on 6 popular TV shows. TVQA consists of 152,545 QA pairs from 21,793 clips, spanning over 460 hours of video. Questions are designed to be compositional in nature, requiring systems to jointly localize relevant moments within a clip, comprehend subtitle-based dialogue, and recognize relevant visual concepts. We provide analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVQA task. The dataset is publicly available at http://tvqa.cs.unc.edu.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Zhe Gan
  4. Yejin Choi
  5. Percy Liang
  6. Caiming Xiong
  7. Yoshua Bengio
  8. Luke S. Zettlemoyer

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Cordelia Schmid
  6. Yi Yang
  7. Ming-Hsuan Yang
  8. Jitendra Malik

In Question Answering & Reading Comprehension

  1. Mohit Bansal
  2. Jie Lei
  3. Licheng Yu
  4. Tamara L. Berg
  5. Jun Liu
  6. Li Xu
  7. Devshree Patel
  8. Ratnam Parikh

Citations in the corpus