Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering

By Vahid Kazemi, Ali Elqursh. 2017

196 citations recorded by the source corpus.

This paper presents a new baseline for visual question answering task. Given an image and a question in natural language, our model produces accurate answers according to the content of the image. Our model, while being architecturally simple and relatively small in terms of trainable parameters, sets a new state of the art on both unbalanced and balanced VQA benchmark. On VQA 1.0 open ended challenge, our model achieves 64.6% accuracy on the test-standard set without using additional data, an improvement of 0.4% over state of the art, and on newly released VQA 2.0, our model scores 59.7% on validation set outperforming best previously reported results by 0.5%. The results presented in this paper are especially interesting because very similar models have been tried before but significantly lower performance were reported. In light of the new results we hope to see more meaningful research on visual question answering in the future.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Question Answering & Reading Comprehension

Citations in the corpus

Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering (2017, methodology): “This approach has been the basis of numerous previous models [16, 20, 38].”
REMIND Your Neural Network to Prevent Catastrophic Forgetting (2019, methodology): “We combine REMIND with two popular VQA algorithms, using a modiﬁed version of the stacked attention network (SAN) [45], [58] for TDIUC, and a simpliﬁed version of the memory attention and control (MAC) [47], [59] network for CLEVR.”
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge (2017, methodology): “In other experiments (not reported in Table 1), we experimented with a tanh activation following the word embed-dings as in [13, 22].”
Pathologies of Neural Models Make Interpretations Difficult (2018, methodology): “The model we use is Show, Ask, Attend, and Answer (Kazemi and Elqursh, 2017).”
Errudite: Scalable, Reproducible, and Testable Error Analysis (2019, methodology): “VQACounting is built on top of SAAA, with increased performance on counting questions.”
Divide and Conquer: Question-Guided Spatio-Temporal Contextual Attention for Video Question Answering (2020, background): “For Image QA, there have been extensive efforts concentrated on (Yang et al. 2016; Xu and Saenko 2016; Wu et al. 2016; Kazemi and Elqursh 2017; Anderson et al. 2018; Gao et al. 2016; Fukui et al. 2016; Ben-Younes et al. 2017) in the last decade.”
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded (2019, background): “For the dominant paradigm of vision-and-language models that compute an explicit question-guided attention over image regions [28, 33, 14, 35, 20, 3], a seemingly straight-”
Are Red Roses Red? Evaluating Consistency of Question-Answering Models (2019, methodology): “For VQA, we evaluate the SAAA baseline (Kazemi and Elqursh, 2017), a recent model with a counting module (Count; Zhang et al.”
Learning to Count Objects in Natural Images for Visual Question Answering (2018, methodology): “This model is based on the work of Kazemi & Elqursh (2017), who outperformed most previous VQA models on the VQA v1 dataset with a simple baseline architecture.”
Did the Model Understand the Question? (2018, background): “Table 1: VQA network (Kazemi and Elqursh, 2017): Accuracy for prefix attacks; original accuracy is 61.”
Learning Visual Question Answering by Bootstrapping Hard Attention (2018, background): “However, only soft attention is used in the majority of Visual QA works [7,8,9,10,11,12,43,44,45,46,47,48,49,50,51,52].”
Scene Text Visual Question Answering (2019, background): “Question and Answers: The ST-VQA dataset comprises 31 , 791 questions.”
MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding (2021, unknown): “Simple baseline methods that only use question understanding (Kazemi and Elqursh 2017) or answer option sentiment analysis (Manjunatha, Saini, and Davis 2019) have been proven to perform surprisingly well on datasets such as VQA (Antol et al. 2015) and VQA2.”
Answering Questions about Data Visualizations using Efficient Bimodal Fusion (2019, methodology): “SANDY used a modiﬁed version of the stacked attention network (SAN) [37, 20], which has been widely used for VQA [20, 4].”
DVQA: Understanding Data Visualizations via Question Answering (2018, background): “In [17], it was shown that upgrading the original SAN’s image features produces state-of-the-art results on VQA 1.0 and 2.0.”
Quantifying and Alleviating the Language Prior Problem in Visual Question Answering (2019, background): “on set. The Question-only [5, 13] isthemodeltrainedwithoutreasoning theimage,anditisforsure that this baseline would arise the language prior problem. The other two HieCoAttn [24] and Strong-baseline [20] are the stateof-the-art models on the VQA 1.0 dataset. For example, if the true answer for a gi”
Cascade Reasoning Network for Text-based Visual Question Answering (2020, unknown): “To better model the relationship of each image region and question word, most studies [14, 22, 32, 47] employ an attention mechanism to learn attended image features for a given question.”
VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions (2018, methodology): “Note that we adopt a simple one-glimpse, one-way attention, as opposed to complex schemes proposed by recent works [5,27,13].”