The Cityscapes Dataset for Semantic Urban Scene Understanding
By Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele. 2016
13,806 citations recorded by the source corpus.
Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.
Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Alan L. Yuille
- Yi Yang
In Evaluation & Benchmarks
- Vladlen Koltun
- Luc Van Gool
- Jitendra Malik
- Trevor Darrell
- Antonio Torralba
- Bernt Schiele
- Kaiming He
- Ross B. Girshick
Citations in the corpus
- Mask R-CNN (2017, methodology): “To do this, we initialize the corresponding 7 categories in Cityscapes from a pre-trained COCO Mask R-CNN model (rider being randomly initialized).”
- Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “5 Besides, MoCo is on par on Cityscapes instance segmentation, and lags behind on VOC semantic segmentation; we show another comparable case on iNatu-ralist [57] in appendix.”
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “We also show state-of-the-art segmentation results on Mapillary Vistas [65], and Cityscapes [23].”
- VOLO: Vision Outlooker for Visual Recognition (2021, methodology): “We report results on two widely-used segmentation benchmarks: Cityscapes [10] and ADE20K [77].”
- Unified Perceptual Parsing for Scene Understanding (2018, background): “The Pyramid Scene Parsing Network (PSPNet) [16] performs spatial pooling at several grid scales and achieves remarkable performance on several segmentation benchmarks [17,18,2].”
- Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “ by iterating over the objects in decreasing depth, ensuring that object parts never occlude whole objects and no object is occluded by its parts. Datasets such as COCO [18], Pascal [10] or Cityscape [7] start by defining a set of object categories of interest. However, when labeling all the objects ”
- Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “Major advances in image understanding tasks [57, 18, 50, 49, 37, 23, 17] have been enabled by the introduction of large scale datasets such as ImageNet [53], MSCOCO [30] or Cityscapes [10].”
- Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression (2019, background): “Due to this appealing property, all performance measures used to evaluate for segmentation [2, 1, 25, 14], object detection [14, 4], and tracking [11, 10] rely on this metric.”
- UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “In this section, we present the experimental results on COCO [23], Cityscapes [6] and our internal dataset.”
- Panoptic Feature Pyramid Networks (2019, methodology): “Cityscapes: Cityscapes [14] is an ego-centric street-scene dataset.”
- Panoptic Segmentation (2018, methodology): “A common annotation practice [6, 25] is to use a group label instead of instance ids for adjacent instances of the same semantic class if accurate delineation of each instance is difficult.”
- Recurrent Neural Networks for Semantic Instance Segmentation (2017, background): “Cityscapes [9]: 5,000 street-view images containing objects of 8 different categories.”
- Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “Finally, we evaluate our algorithm on the Cityscapes road-scene understanding dataset [8].”
- Shape-aware Instance Segmentation (2016, methodology): “The Cityscapes dataset [4] consists of 9 object categories for instance-level semantic labeling.”
- InstanceCut: From Edges to Instances with MultiCut (2016, methodology): “Current state-of-the-art methods report around 78% mean Intersection-over-Union (IoU) for the CityScapes dataset [13] and about 82% for the PASCAL VOC2012 challenge [18].”
- Projected GANs Converge Faster (2021, methodology): “On the large datasets, the baselines only outperform Projected GAN in precision on FFHQ, Cityscapes, and LSUN Church (Table 4).”
- Semantic Image Synthesis With Spatially-Adaptive Normalization (2019, methodology): “In Figure 19 and 20, we show additional synthesis re-sults from the proposed method on the ADE20K-outdoor and Cityscapes datasets with comparison to those from the CRN [7], SIMS [35], and pix2pixHD [40] methods.”
- COCO-Stuff: Thing and Stuff Classes in Context (2016, methodology): “More recent large-scale datasets like Barcelona [50], LM+SUN [52], PASCAL Context [38], Cityscapes [13] and ADE20K [63] annotate tens of thousands of images with hundreds of classes.”