LabelMe: A Database and Web-Based Tool for Image Annotation

By Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman. 2008

4,545 citations recorded by the source corpus.

Abstract We seek to build a large collection of images with ground truth labels to be used for object detection and recognition research. Such data is useful for supervised learning and quantitative evaluation. To achieve this, we developed a web-based tool that allows easy image annotation and instant sharing of such annotations. Using this annotation tool, we have collected a large dataset that spans many object categories, often containing multiple instances over a wide variety of images. We quantify the contents of the dataset and compare against existing state of the art datasets used for object recognition and detection. Also, we show how to extend the dataset to automatically enhance object labels with WordNet, discover object parts, recover a depth ordering of objects in a scene, and increase the number of labels using minimal user supervision and images from the web.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Xiaogang Wang
4. Trevor Darrell
5. Ming-Hsuan Yang
6. Jitendra Malik
7. Andrew Zisserman
8. Bernt Schiele

In Evaluation & Benchmarks

1. Trevor Darrell
2. Antonio Torralba
3. Li Fei-Fei
4. Jitendra Malik
5. Luc Van Gool
6. Bernt Schiele
7. Abhinav Kumar Gupta
8. Alan L. Yuille

Citations in the corpus

• ImageNet Large Scale Visual Recognition Challenge (2014, background): “LabelMe (Russell et al., 2007) contains general photographs with multiple objects per image.”
• Microsoft COCO: Common Objects in Context (2014, methodology): “An object’s spatial location can be defined coarsely using a bounding box [2] or with a precise pixel-level segmentation [14], [15], [16].”
• Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “ous background regions, e.g., grass, sky, or 3) object part, which is a component of some existing object instance which has some functional meaning, such as head or leg. Images come from the LabelMe [30], SUN datasets [34], and Places [38] and were selected to cover the 900 scene categories defined ”
• LVIS: A Dataset for Large Vocabulary Instance Segmentation (2019, neutral): “9 and [29, 25, 24, 27]) .”
• 3D Instance Segmentation of MVS Buildings (2021, methodology): “Similar to most 2D annotation tools [48] which automatically extract the pixels of an object by marking the closed boundary polygon of the object.”
• Extreme Clicking for Efficient Object Annotation (2017, background): “Quality control is a common process when crowd-sourcing image annotations [4, 36, 43, 53, 56, 60, 62, 66, 70].”
• Scene Parsing through ADE20K Dataset (2017, methodology): “Images come from the LabelMe [27], SUN datasets [32], and Places [35] and were selected to cover the 900 scene”
• HMDB: A large video database for human motion recognition (2011, background): “Current action recognition databases contain on the order of ten different action categories collected under fairly controlled conditions.”
• Material recognition in the wild with the Materials in Context Database (2014, neutral): “For segmentation or scene parsing tasks, per-pixel segmentations are required [22, 8].”
• ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes (2017, background): “The semantic seg-mentations are more than an order of magnitude larger than any previous RGB-D dataset.”
• Undoing the Damage of Dataset Bias (2012, methodology): “For example, if Caltech101 is the current unseen test set, then the model is trained on PASCAL2007, LabelMe, and SUN09.”
• SUN Database: Exploring a Large Collection of Scene Categories (2014, methodology): “To provide data for research of objects in scenes, using LabelMe (Russell et al. 2008), we have also labeled objects”
• Picture: A probabilistic programming language for scene perception (2015, methodology): “We evaluated our program on a dataset of humans per­ forming a variety of poses, which was aggregated from KTH [39] and LabelMe [38] images with significant oc­ clusion in the "person sitting"(around 50 total images).”
• Using Multiple Segmentations to Discover Objects and their Extent in Image Collections (2006, methodology): “Caltech [9] 4,090 4 + background MSRC [27] 4,325 23 object and scene categories LabelMe [18] 1,554 cars, buildings, trees”