Where to Look: Focus Regions for Visual Question Answering

By Kevin J. Shih, Saurabh Singh, Derek Hoiem. 2015

487 citations recorded by the source corpus.

We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method maps textual queries and visual features from various regions into a shared space where they are compared for relevance with an inner product. Our method exhibits significant improvements in answering questions such as "what color," where it is necessary to evaluate a specific location, and "what room," where it selectively identifies informative image regions. Our model is tested on the recently released VQA [1] dataset, which features free-form human-annotated questions and answers.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Dacheng Tao
  4. Trevor Darrell
  5. Yoshua Bengio
  6. Zhe Gan
  7. Percy Liang
  8. Devi Parikh

In Computer Vision and Pattern Recognition

  1. Dacheng Tao
  2. Luc Van Gool
  3. Xiaogang Wang
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Bernt Schiele
  7. Yi Yang
  8. Jitendra Malik

In Question Answering & Reading Comprehension

  1. Anton van den Hengel
  2. Christopher Kanan
  3. Kushal Kafle
  4. Damien Teney
  5. Mohit Bansal
  6. Qi Wu
  7. Chunhua Shen
  8. Jie Lei

Citations in the corpus