Scene Parsing through ADE20K Dataset

By Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba. 2017

3,986 citations recorded by the source corpus.

Scene parsing, or recognizing and segmenting objects and stuff in an image, is one of the key problems in computer vision. Despite the communitys efforts in data collection, there are still few image datasets covering a wide range of scenes and object categories with dense and detailed annotations for scene parsing. In this paper, we introduce and analyze the ADE20K dataset, spanning diverse annotations of scenes, objects, parts of objects, and in some cases even parts of parts. A scene parsing benchmark is built upon the ADE20K with 150 object and stuff classes included. Several segmentation baseline models are evaluated on the benchmark. A novel network design called Cascade Segmentation Module is proposed to parse a scene into stuff, objects, and object parts in a cascade and improve over the baselines. We further show that the trained scene parsing networks can lead to applications such as image content removal and scene synthesis1.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “To evaluate the effectiveness of the CCNet, we carry out comprehensive experiments on the Cityscapes dataset [10], the ADE20K dataset [44], and the COCO dataset [23].”
Bag of Tricks for Image Classification with Convolutional Neural Networks (2018, methodology): “We use Fully Convolutional Network (FCN) [17] as the baseline approach and evaluate it on the ADE20K [34] dataset.”
Unified Perceptual Parsing for Scene Understanding (2018, methodology): “Various image datasets are constructed only for speciﬁc task, such as ADE20K for scene parsing [2], the Describe Texture Dataset (DTD) for texture recognition [4], and OpenSurfaces for material and surface recognition [6].”
Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “ical Turk). Furthermore, the data consistency and quality are much higher than that of external annotators. Fig. 1 shows examples from our dataset. The preliminary result of this work is published at [39]. Compared to the previous conference paper, we include more description of the dataset, more ba”
Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “ADE20k [60] is a scene parsing dataset, containing 20 210 training, 2 000 validation samples, annotated with a dictionary of 150 labels.”
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression (2019, background): “Due to this appealing property, all performance measures used to evaluate for segmentation [2, 1, 25, 14], object detection [14, 4], and tracking [11, 10] rely on this metric.”
Panoptic Feature Pyramid Networks (2019, methodology): “For instance segmentation, the region-based Mask R-CNN [24] with a Feature Pyramid Network (FPN) [36] backbone has been used as a foundation for all top entries in recent recognition challenges [37, 60, 43].”
Panoptic Segmentation (2018, methodology): “This includes the Cityscapes [6], ADE20k [54], and Mapillary Vistas [35] datasets.”
Projected GANs Converge Faster (2021, methodology): “We compare on LSUN cat and horse [84], ADE indoor (a subset of ADE [91] proposed in [3]), the full Oxford ﬂowers dataset with 8k images [56], KITTI-ﬁsheye (a subset of KITTI-360 [46], consisting of ﬁsheye images), STL-10 [8], CUB200 [80], Stanford Dogs [36], and Stanford Cars [42].”
Semantic Image Synthesis With Spatially-Adaptive Normalization (2019, methodology): “In Figure 19 and 20, we show additional synthesis re-sults from the proposed method on the ADE20K-outdoor and Cityscapes datasets with comparison to those from the CRN [7], SIMS [35], and pix2pixHD [40] methods.”
COCO-Stuff: Thing and Stuff Classes in Context (2016, methodology): “One reason for the recent success of deep learning methods is the advent of large-scale datasets [43, 29, 63].”
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “We report additional qualitative results on ADE20K dataset [61] and Helen Face dataset [30, 48].”
A Simple Framework for Open-Vocabulary Segmentation and Detection (2023, methodology): “During fine-tuning, we use 512 × 1024 for Cityscapes [6] and 640× 640 for ADE20K [57] by default.”
Generalized Decoding for Pixel, Image, and Language (2022, methodology): “[98] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba.”
Multiview Compressive Coding for 3D Reconstruction (2023, background): “Popular 2D visual tasks, such as object classification [8] and segmentation [33, 83], aim to recognize them on the image plane.”
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, methodology): “(a) an input image from ADE20K [61] (upper row) with the ground truth object masks overlaid (lower row).”
FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “We also report the performance of our model trained on SBD [15], and a large combined dataset [2, 7, 13, 24, 26, 39, 45, 48].”
Masked-attention Mask Transformer for Universal Image Segmentation (2021, methodology): “We compare Mask2Former with state-of-the-art models for semantic segmentation on the ADE20K [65] dataset in Table 3.”
Per-Pixel Classification is Not All You Need for Semantic Segmentation (2021, methodology): “We evaluate MaskFormer on five semantic segmentation datasets with various numbers of categories: Cityscapes [15] (19 classes), Mapillary Vistas [34] (65 classes), ADE20K [55] (150 classes), COCOStuff-10K [3] (171 classes), and ADE20K-Full [55] (847 classes).”