LabelMe: Online Image Annotation and Applications

By Antonio Torralba, Bryan C. Russell, Jenny Yuen. 2010

401 citations recorded by the source corpus.

Central to the development of computer vision systems is the collection and use of annotated images spanning our visual world. Annotations may include information about the identity, spatial extent, and viewpoint of the objects present in a depicted scene. Such a database is useful for the training and evaluation of computer vision systems. Motivated by the availability of images on the Internet, we introduced a web-based annotation tool that allows online users to label objects and their spatial extent in images. To date, we have collected over 400 000 annotations that span a variety of different scene and object classes. In this paper, we show the contents of the database, its growth over time, and statistics of its usage. In addition, we explore and survey applications of the database in the areas of computer vision and computer graphics. Particularly, we show how to extract the real-world 3-D coordinates of images in a variety of scenes using only the user-provided object annotations. The output 3-D information is comparable to the quality produced by a laser range scanner. We also characterize the space of the images in the database by analyzing 1) statistics of the co-occurrence of large objects in the images and 2) the spatial layout of the labeled images.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

See No Evil, Say No Evil: Description Generation from Densely Labeled Images (2014, background): “Figure 1 shows more examples for objects 3 We modiﬁed LabelMe (Torralba et al., 2010). in a labeled image.”
Furniture-geek: Understanding fine-grained furniture attributes from freely associated text and tags (2014, background): “The vision community has relied on crowdsourcing for human supervision in several image understanding tasks like scene understanding [28], object recognition [ 10] and human pose estimation [5].”
Scene Recognition with CNNs: Objects, Scales and Dataset Bias (2016, methodology): “Fortunately, both ImageNet and SUN have a fraction of images with region annotations and labels, so we can collect some relevant statistics and compare their distributions (we used the LabelMe toolbox[14]).”
3DNN: 3D Nearest Neighbor (2014, background): “2010), for which there exist LabelMe annotations (Torralba et al. 2010).”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, methodology): “We also trained SegNet on a larger dataset of road scenes collected from various publicly available datasets [3], [53], [54] and show that this leads to a large improvement in accuracy.”