Labeling images with a computer game

By Luis von Ahn, Laura A. Dabbish. 2004

2,580 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Human-Computer Interaction

Citations in the corpus

ImageNet Large Scale Visual Recognition Challenge (2014, neutral): “A related line of work is to obtain annotations through well-designed games, e.g. (von Ahn and Dabbish, 2005).”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, background): “In particular, standard image annotation datasets used by Makadia et al. (2008); Guillaumin et al. (2009); Rasiwasia and Vasconcelos (2007); Verma and Jawahar (2012) – namely, Corel5K (Duygulu et al., 2002), ESP Game (von Ahn and Dabbish, 2004), and IAPR-TC (Grubinger et al., 2006) – have only two v”
LabelMe: A Database and Web-Based Tool for Image Annotation (2008, background): “Note that for applications where it is important to have uniform prior distribitions of object locations and sizes, we suggest cropping and rescaling each image randomly.”
Grounding Action Descriptions in Videos (2013, background): “A large multimodal resource combining language and visual information resulted from the ESP game (von Ahn and Dabbish, 2004).”
Global-and-local attention networks for visual recognition (2018, background): “Online games constitute an efﬁcient way to collect human ground-truth data [26, 32–35].”
Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games (2017, background): “ intelligence embraces social and collective wisdom [9], which lays an essential foundation for reaching the grand goal of Artiﬁcial General Intelligence (AGI) [10]. As demonstrated by crowd sourcing [11], aggregating efforts collectively from the public would solve the problem which is otherwise un”
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue (2016, methodology): “The first ESP game [4] was developed to collect image tags, and was later extended to Peekaboom [5] to gather object segmentations.”
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent (2017, positive): “There are some systems that have attempted to apply competitive, collaborative and/or gamiﬁcation strategies to collect data, notably the ESP game (Von Ahn & Dabbish, 2004), which is an image annotation tool where users are paired and have to “read each others mind” to agree on the contents of an…”
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021, background): “Gamiﬁca-tion, in the style of the ESP game or ZombiLingo (Von Ahn and Dabbish, 2004; Fort et al., 2014), promises to offer free human labor, but at the cost of the expert time needed to reﬁne the task deﬁ-nition into a game that is widely enjoyable.”
Multi-Label Prediction via Compressed Sensing (2009, methodology): “Image data.1 The first data set was collected by the ESP Game [26], an onlinegame in which players ultimately provide word tags for a diverse set of web images.”
Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning (2019, methodology): “Some widely used databases include Corel 5k [16] (including 4,999 images with 260 object categories), ESP Game [17] (including 20,770 images with 268 categories), IAPRTC-12 [18] (including 19,627 images with 291 categories), NUSWIDE [19] (including 270K images with 81 categories), MS COCO [20] (incl”
Building a database of 3D scenes from user annotations (2009, methodology): “Finally, we use a prior on the camera height, with mean 1.7 meters and standard deviation 0.5 meters.”