Answer Them All! Toward Universal Visual Question Answering Models

By Robik Shrestha, Kushal Kafle, Christopher Kanan. 2019

87 citations recorded by the source corpus.

Visual Question Answering (VQA) research is split into two camps: the first focuses on VQA datasets that require natural image understanding and the second focuses on synthetic datasets that test reasoning. A good VQA algorithm should be capable of both, but only a few VQA algorithms are tested in this manner. We compare five state-of-the-art VQA algorithms across eight VQA datasets covering both domains. To make the comparison fair, all of the models are standardized as much as possible, e.g., they use the same visual features, answer vocabularies, etc. We find that methods do not generalize across the two domains. To address this problem, we propose a new VQA algorithm that rivals or exceeds the state-of-the-art for both domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

REMIND Your Neural Network to Prevent Catastrophic Forgetting (2019, background): “, attention-based [6,46,78], compositional [7,28], bi-modal fusion [10,19,71]) and it can be applied to similar tasks like image captioning [12] and referring expression recognition [43,63,67].”
Learning by Abstraction: The Neural State Machine (2019, methodology): “24.96 HAN [59] 28.65 GVQA [3] 31.30 RAMEN [73] 39.21 BAN [46] 39.31 MuRel [15] 39.54 ReGAT [52] 40.42 NSM 45.80 First, we measure the performance on VQA-CP [3], which provides a new split of the VQA2 dataset [27], where the answer distribution is kept different between the training and the test…”
Challenges and Prospects in Vision and Language Research (2019, unknown): “, 2018), with very few exceptions (Anderson et al., 2018; Kim et al., 2018; Shrestha et al., 2019).”
RUBi: Reducing Unimodal Biases in Visual Question Answering (2019, methodology): “Models with ** have been trained by [45]. the second best approach UpDn + Q-Adv + DoE only achieves a +1.43 gain in overall accuracy over their baseline UpDn.”
Interpretable Visual Reasoning via Probabilistic Formulation Under Natural Supervision (2020, background): “We compare with two recently proposed reasoning models with multi-step attention and fusion strategy (MUTAN [6], MuRel [8]), three models focus on relation reasoning (QCG [43], Dyna Tree [52], DFAF [14]) and RAMEN [50] that claims to work on both real-world and synthetic datasets.”
TRAR: Routing the Attention Spans in Transformer for Visual Question Answering (2021, background): “The rapid development of VQA has been supported by the emergence of various benchmark datasets [1, 3, 14, 28, 29, 32, 52] and methods [13, 59, 66, 72, 54, 46, 77, 78, 79, 80].”
Answering Questions about Data Visualizations using Efficient Bimodal Fusion (2019, methodology): “Jointly modulating visual features using vision and language features can allow models to learn richer features for downstream tasks [26, 28, 34].”
CogVLM: Visual Expert for Pretrained Language Models (2023, unknown): “To comprehensively investigate the proposed model on specific topics and question types, we further conduct extensive experiments on a representative benchmark, TDIUC [Kafle and Kanan, 2017].”
RSVQA: Visual Question Answering for Remote Sensing Data (2020, background): “Indeed, it has been shown in [28] that VQA models trained”
In Defense of Grid Features for Visual Question Answering (2020, background): “We hope that our discovery that grid features also work well for the general VQA task can bridge the gap between these two lines of work [34].”
VQD: Visual Query Detection In Natural Scenes (2019, background): “While the state-of-the-art has rapidly increased on both synthetic and natural image VQA datasets, many models do not generalize across datasets (Shrestha et al., 2019).”