Visual Madlibs: Fill in the Blank Description Generation and Question Answering

By Licheng Yu, Eunbyung Park, Alexander C. Berg, Tamara L. Berg. 2015

146 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Learning Visual Features from Large Weakly Supervised Data (2015, methodology): “In future work, we aim to combine our weakly supervised vision models with a language model such as word2vec [31] to perform, for instance, visual question answering [3, 58].”
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge (2019, background): “Madlibs [51].”
VQA: Visual Question Answering (2015, background): “[47] introduced a dataset of 10k images and prompted captions that describe specific aspects of a scene (e.”
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (2016, background): “have proposed visual question answering datasets [6,31,35, 41,14,57,48,46] and models [13,34,5,54,33,36,59,56,55, 52,45,27,38,20,53,43,23].”
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering (2017, background): “Most existing VQA datasets are image-centric, e.g ., DAQUAR [25], abstract scenes [24], VQA [2], Visual Madlibs [41], DAQUAR-Consensus [26], FM-IQA [11], COCO-QA [29], and Visual7W [43].”
TGIF: A New Dataset and Benchmark on Animated GIF Description (2016, methodology): “The VQA [1] and the Visual Madlibs [44] datasets are released for image captioning and visual question answering.”
FVQA: Fact-Based Visual Question Answering (2016, methodology): “They have contributed valuable large-scale data for training neural-network based VQA models and introduced various question types, and tasks, from global association between QA pairs and images [5], [6], [9] to grounded QA in image regions [10]; from free-from answer generation [5], [7], [9], [10] ”
Ask Your Neurons: A Deep Learning Approach to Visual Question Answering (2016, background): “Similarly to Visual Madlibs (Yu et al, 2015), Visual7W also contains multiple-choice answers.”
VQD: Visual Query Detection In Natural Scenes (2019, background): “…few years, a large amount of work has been done at the intersection of computer vision and natural language understanding, including visual madlibs (Yu et al., 2015; Tommasi et al., 2018), captioning (Farhadi et al., 2010; Kulkarni et al., 2013; Johnson et al., 2016; Liu et al., 2018), and image…”
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions (2016, background): “Therefore, there has been a movement toward more focused tasks, such as visual question answering [2, 30], and referring expression generation and comprehension which involve specific regions/objects within an image (discussed below).”
RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes (2018, background): “…in computer vision closely related to RC is Visual Question Answering (VQA) in which context takes the form of an image in the comprehension task, where recent datasets have also been compiled, such as (Antol et al., 2015; Yu et al., 2015; Johnson et al., 2017; Goyal et al., 2017), to name a few.”
IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning (2018, background): “One possible answer would be to measure intuitive physics through real-worlds applications like Visual Question Answering (VQA), object tracking or action planning (see Figure 1).”
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge (2022, background): “Many datasets for this task have been proposed [2,9,23,30,40,46,52,56] but most of these do not require much outside knowledge or reasoning, often focusing on recognition tasks such as classification, attribute detection, and counting.”
Crowdsourcing in Computer Vision (2016, unknown): “[Yu et al., 2015] collect answers that can be used to fill blanks in descriptions of images.”