VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases

By Fereshteh Sadeghi, Santosh Kumar Divvala, Ali Farhadi. 2015

124 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Multimodal Learning

Citations in the corpus

Solving General Arithmetic Word Problems (2016, background): “A recent work (Sadeghi et al., 2015) tries to answer science questions by visually verifying relations from images.”
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge (2019, background): “Several knowledge bases have been created using visual data or for visual reasoning tasks [53, 9, 11, 39, 56, 55].”
VQA: Visual Question Answering (2015, background): “[40] and [44] use visual information to assess the plausibility of common sense assertions.”
Grounding of Textual Phrases in Images by Reconstruction (2015, background): “[36] attempts to localize relation phases of type Subject-Verb-Object at a large scale in order to verify their correctness, while relying on detectors from [8].”
Learning Language-Visual Embedding for Movie Understanding with Natural-Language (2016, background): “There are also abundant evidence [19,27] that jointly learning from language and visual modalities can be mutually beneficial.”
Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources (2015, background): “Using visually-sourced information is a promising approach to solving this problem [16, 24], but has a way to go before it might be usefully applied within our approach.”
The More You Know: Using Knowledge Graphs for Image Classification (2016, background): “Learning knowledge graphs [38, 4, 29] and using graphs for visual reasoning [39, 20] has recently been of interest to the vision community.”
Modeling Relationships in Referential Expressions with Compositional Modular Networks (2016, background): “Aside from localizing relationship expressions, [30] generates image descriptions using a recurrent network with attention over image feature grids, and [25, 31] learns to extract visual relation knowledge from images.”
Weakly-Supervised Learning of Visual Relations (2017, background): “Triplet learning has been addressed in various tasks such as mining typical relations (knowledge extraction) [7, 43, 52, 54], reasoning [21, 35, 45], object detection [17, 44], image retrieval [23] or fact retrieval [11].”
Neural Motifs: Scene Graph Parsing with Global Context (2017, neutral): “Much of the context we derive could be interpreted as commonsense priors, which have commonly been extracted using auxiliary means [59, 39, 5, 49, 55].”
Video Relationship Reasoning Using Gated Spatio-Temporal Energy Graph (2019, methodology): “Conditional Random Fields (CRFs) have been popularly used to model the statistical dependencies among predictions in images [10, 43, 27, 25, 4] and videos [23, 31].”
Leveraging Visual Question Answering for Image-Caption Ranking (2016, background): “[41,42] evaluate the plausibility of commonsense assertions by verifying them on collections of abstract scenes and real images, respectively, to leverage the visual common sense in those collections.”
Verb Physics: Relative Physical Knowledge of Actions and Objects (2017, background): “Thus, our work provides unique value to complement recent attempts to acquire commonsense knowledge from web images (Izadinia et al., 2015; Bagherinezhad et al., 2016; Sadeghi et al., 2015).”
Video Fill In the Blank Using LR/RL LSTMs with Spatial-Temporal Attentions (2017, background): “Another binary answering problem which verifies existence of a relation between two concepts (for example: dogs eat ice cream) is proposed in [25], which uses CNNs to verify the relation.”
Building a Large-scale Multimodal Knowledge Base System for Answering Visual Queries (2015, background): “In vision, a series of work has focused on extracting relational knowledge from visual data [5, 39, 60].”
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA (2020, background): “There have been a number of knowledge bases proposed and used for knowledge-based reasoning, both for language-only and multi-modal tasks [88, 14, 17, 66, 91, 90, 10, 54, 37].”
Human-Centered Artificial Intelligence and Machine Learning (2019, unknown): “, 2015), images (Sadeghi et al., 2015), and video.”