Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering

2018

151 citations recorded by the source corpus.

Citations in the corpus

• KAT: A Knowledge Augmented Transformer for Vision-and-Language (2021, neutral): “Several methods explore aggregating the external knowledge either in the form of structured knowledge graphs (Garderes et al., 2020; Narasimhan et al., 2018; Li et al., 2020b; Wang et al., 2017a,b) or unstructured knowledge bases (Marino et al., 2021; Wu et al., 2021; Luo et al., 2021).”
• OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge (2019, neutral): “Given the question, image, and Wikipedia sentences, there is now enough information to answer the question: Teddy Roosevelt! More recent research has started to look at how to incorporate knowledge-based methods into VQA [29, 30, 36, 37].”
• A Comprehensive Survey on Graph Neural Networks (2019, neutral): “It includes human-object interaction [144], few-shot image classification [145], [146], [147], semantic segmentation [148], [149], visual reasoning [150], and question answering [151].”
• Deep Multimodal Data Fusion (2024, unknown): “…there are an increasing number of applications based on GNNs addressing the multimodal problems related to graph data, e.g., VQA task [77, 95, 100, 129, 190, 214, 218], image captioning task [42, 182, 182, 208–210], cross-modal retrieval task [30, 204, 217], RGB-depth scene classification task…”
• Explainability Techniques for Graph Convolutional Networks (2019, unknown): “…Zitnik et al., 2018), physics (Battaglia et al., 2016; Chang et al., 2017; Gilmer et al., 2017; Watters et al., 2017; Sanchez-Gonzalez et al., 2018), visual recognition (Qi et al., 2017; 2018; Narasimhan et al., 2018), and natural language processing (Bastings et al., 2017; Beck et al., 2018).”
• A Memorizing and Generalizing Framework for Lifelong Person Re-Identification (2023, unknown): “The techniques of GNN are applied to various tasks, such as semi-supervised classification [57], visual question answering [59], image captioning [60], shape completion [61] and point cloud segmentation [62].”
• REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering (2022, unknown): “Recent studies [29, 28, 20, 19, 35, 31, 17, 32, 5] integrate different knowledge from various external knowledge resources, e.”
• Coarse-to-Fine Reasoning for Visual Question Answering (2021, unknown): “Current methods focus on utilizing visual information [5,22,33,35,38,40,42,44,53] without considering if the supporting information is useful or not [16].”
• Object Relational Graph With Teacher-Recommended Learning for Video Captioning (2020, unknown): “Some researches have shown that visual relational reasoning is effective for computer vision tasks, such as Image Captioning [49, 46], VQA [23, 22, 18] and Action Recognition [41, 42].”
• Factor Graph Attention (2019, unknown): “Image captioning [36, 56, 24, 50, 7, 4, 13] was an instrumental language+vision task, followed by visual question answering [33, 42, 25, 34, 41, 5, 15, 59, 23, 3, 9, 14, 46, 55, 42, 54, 39, 40, 43] and visual question generation [41, 38, 22, 49, 28, 6].”
• Multi-Modal Answer Validation for Knowledge-Based VQA (2021, unknown): “Narasimhan and Schwing (2018) propose to retrieve relevant facts from a knowledge base.”
• Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering (2020, unknown): “[ Narasimhan et al. , 2018 ] applies GCN on the fact graph where each node is represented by the fixed form of image-question-entity embedding.”
• Hypergraph Convolution and Hypergraph Attention (2019, unknown): “…(Hamilton et al., 2017) and GCPN (You et al., 2018a), and apply them to other domain-specific applications, e.g. , 3D shape analysis (Boscaini et al., 2016), visual question answering (Narasimhan et al., 2018), chemistry (Gilmer et al., 2017; Liu et al., 2018) and NP-hard problems (Li et al., 2018).”
• GraphQA: protein model quality assessment using graph convolutional networks (2020, unknown): “, 2018), visual scene understanding (Narasimhan et al., 2018) and natural language understanding (Kipf & Welling, 2017).”
• BERT Representations for Video Question Answering (2020, unknown): “For example, in Spatial Memory Network [42], neuron activations from different regions of the image are stored in a memory; in [29], an entity graph based question answering is proposed, where graph convolutional network is used to simulate the reasoning about the correct answer; in interactive…”
• Structured Multimodal Attentions for TextVQA (2020, unknown): “Narasimhan et al. [28] perform finer relation exploration for factual-VQA task [14] by taking into account a list of facts via Graph Convolution Networks (GCN) for correct answer selection.”
• Reveal: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory (2022, unknown): “One line of research uses explicit knowledge either from structured knowledge graphs [11, 30, 43] or unstructured text corpora [26, 27, 45].”
• PromptCap: Prompt-Guided Task-Aware Image Captioning (2022, unknown): “Prior works [13, 36, 63, 72, 40, 41, 17, 18, 12] investigate leveraging knowledge”