Multi-level Attention Networks for Visual Question Answering

By Dongfei Yu, Jianlong Fu, Tao Mei, Yong Rui. 2017

228 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhe Gan
2. Jingjing Liu
3. Linjie Li
4. Mohit Bansal
5. Xiaogang Wang
6. Hongsheng Li
7. Jianfeng Gao
8. Yu Cheng

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Xiaogang Wang
3. Dacheng Tao
4. Yi Yang
5. Ming-Hsuan Yang
6. Trevor Darrell
7. Cordelia Schmid
8. Wanli Ouyang

In Multimodal Learning

1. Zhe Gan
2. Linjie Li
3. Jingjing Liu
4. Lijuan Wang
5. Mohit Bansal
6. Yu Cheng
7. Zicheng Liu
8. Ivan Laptev

Citations in the corpus

• Attention on Attention for Image Captioning (2019, background): “There are other attention mechanisms such as: spatial and channel-wise attention [6], adaptive attention [27], stacked attention [46], multi-level attention [51], multi-head attention and self-attention [35].”
• Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning (2021, methodology): “Early works use CNN classification models pre-trained on ImageNet to extract visual features [8, 24, 26, 28, 46, 47].”
• Semantics Disentangling for Text-To-Image Generation (2019, methodology): “Inspired by [39, 8, 7, 37], the visual-semantic embedding (VSE) module is adopted for mutual fusion of word features and visual features, as shown in Fig.”
• Motion-Appearance Co-memory Networks for Video Question Answering (2018, background): “[38] developed a semantic attention mechanism to select high-level question-related concepts.”
• Semantic Segmentation of Underwater Imagery: Dataset and Benchmark (2020, background): “Moreover, this semantic knowledge is potentially useful for learning spatio-temporal attention modeling and visual question answering [47], [48], i.”
• Can you fool AI with adversarial examples on a visual Turing test? (2017, unknown): “Semantic attention has been explored by [58].”
• Action Modifiers: Learning From Adverbs in Instructional Videos (2019, background): “Several problems are at the intersection between language and video: captioning [24, 38, 55, 59], retrieval [9, 16, 21, 31, 33, 52, 54] and visual question answering [15, 56, 57, 61].”
• Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks (2017, methodology): “Most of the present methods solve VQA by jointly learning interactions and performing inference over the question and image contents based on the recent success of deep learning [19, 2, 22, 9, 8], which can be further improved by introducing the attention mechanisms [34, 32, 31, 17, 33, 1].”
• TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays (2018, background): “In vision of visual captioning, our work is closed to [37, 33, 29, 38, 27].”
• Progressive Attention Networks for Visual Attribute Prediction (2016, background): “Image caption generation and visual question answering are often benefited from this attention mechanism [1, 22, 23, 24, 25, 27].”
• CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval (2019, background): “Different types of interactions have been explored in language and vision tasks beyond text-image retrieval [32, 2, 20, 35, 12, 29, 21, 18, 19].”
• R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering (2018, methodology): “y semantic attention to learn semantic knowledge like concepts and attributes. It’s shown that our proposed multistep semantic attention network (denoted as RelAtt) improves the state-of-the-art MLAN [41] model from 64.60% to 65.69% on the open-ended task, and from 64.80% to 69.60% on the multi-choi”
• Relation-Aware Graph Attention Network for Visual Question Answering (2019, methodology): “To make the model more interpretable, some literature [30, 57, 29, 52, 53, 51] also exploited high-level semantic information in the image, such as attributes, captions and visual relation facts.”
• Multi-interaction Network with Object Relation for Video Question Answering (2019, background): “[37] develop a semantic attention mechanism to select high-level question related concepts.”
• Spatiotemporal-Textual Co-Attention Network for Video Question Answering (2019, background): “ImageQA has drawn significant research interests recently [2, 3, 8, 12, 30, 31, 39, 51, 52, 60].”
• Improved Fusion of Visual and Language Representations by Dense Symmetric Co-attention for Visual Question Answering (2018, methodology): “[31] utilized attention to extract not only spatial information but also language concept of the image.”
• Fooling Vision and Language Models Despite Localization and Attention Mechanism (2017, unknown): “Semantic attention has been explored by [78].”