VQD: Visual Query Detection In Natural Scenes

By Manoj Acharya, Karan Jariwala, Christopher Kanan. 2019

19 citations recorded by the source corpus.

We propose a new visual grounding task called Visual Query Detection (VQD). In VQD, the task is to localize a variable number of objects in an image where the objects are specified in natural language. VQD is related to visual referring expression comprehension, where the task is to localize only one object. We propose the first algorithms for VQD, and we evaluate them on both visual referring expression datasets and our new VQDv1 dataset.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

REMIND Your Neural Network to Prevent Catastrophic Forgetting (2019, neutral): “atility, we pioneered streaming VQA and established strong baselines. It would be interesting to extend this to streaming chart question answering [37,38,40], object detection, visual query detection [2], and other problems in vision and language [39]. Acknowledgements. This work was supported in pa”
RODEO: Replay for Online Object Detection (2020, neutral): “Our method can be easily extended for the VQD task by modifying the object detector to output only the boxes relevant to the language query.”
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (2019, methodology): “We use a softmax function to squash all the raw scores into the interval of [0, 1] as the localization confidences S = {si} ∈ RM×1 for the proposed M bounding boxes.”
Selective Replay Enhances Learning in Online Continual Analogical Reasoning (2021, background): “Some problems requiring these additional components include visual question answering [8, 44, 56], referring expression recognition [45, 68], visual query detection [3], and image captioning [11].”
Challenges and Prospects in Vision and Language Research (2019, background): “To counter this, Acharya et al. (2019) proposed visual query detection (VQD), a form of goal-directed object detection, where the query can have 0–15 valid boxes making the task more difficult and more applicable to real-world applications.”
Answering Questions about Data Visualizations using Efficient Bimodal Fusion (2019, methodology): “VQA has been explored in open-ended [7, 18], counting [2], multiple choice [7, 27], and pointing type setups [43, 1].”
Answer Them All! Toward Universal Visual Question Answering Models (2019, background): “Another interesting avenue is to combine VQA with related tasks like visual query detection [1].”