Are we Asking the Right Questions in MovieQA?

By Bhavan A. Jasani, Rohit Girdhar, Deva Ramanan. 2019

17 citations recorded by the source corpus.

Joint vision and language tasks like visual question answering are fascinating because they explore high-level understanding, but at the same time, can be more prone to language biases. In this paper, we explore the biases in the MovieQA dataset and propose a strikingly simple model which can exploit them. We find that using the right word embedding is of utmost importance. By using an appropriately-trained word embedding, about half the Question-Answers (QAs) can be answered by looking at the questions and answers alone, completely ignoring narrative context from video clips, subtitles, and movie scripts. Compared to the best published papers on the leaderboard, our simple question+answer only model improves accuracy by 5% for video + subtitle category, 5% for subtitle, 15% for DVS and 6% higher for scripts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “ating geolocation [81,82], object recognition with local features only [36], reasoning based on single cue words [59,14] or answering questions about a movie without ever showing the movie to a model [83]. Importantly, this is not meant to imply that DNNs cannot acquire high-level abilities. They ce”
Condensed Movies: Story Based Retrieval with Contextual Embeddings (2020, background): “gent system, and might help avoid some of the biases present inherently in VQA datasets [4].”