Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing

By Magnus Wrenninge, Jonas Unger. 2018

233 citations recorded by the source corpus.

We introduce Synscapes -- a synthetic dataset for street scene parsing created using photorealistic rendering techniques, and show state-of-the-art results for training and validation as well as new types of analysis. We study the behavior of networks trained on real data when performing inference on synthetic data: a key factor in determining the equivalence of simulation environments. We also compare the behavior of networks trained on synthetic data and evaluated on real-world data. Additionally, by analyzing pre-trained, existing segmentation and detection models, we illustrate how uncorrelated images along with a detailed set of annotations open up new avenues for analysis of computer vision systems, providing fine-grain information about how a model's performance changes according to factors such as distance, occlusion and relative object orientation.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, background): “No existing synthetic dataset or simulator addresses all of these limitations, including those that target outdoor scene understanding [8, 20, 23, 34, 38, 41, 55, 56, 58, 60, 65, 76, 77].”
Synthetic Data for Deep Learning (2019, methodology): “Another data point is provided by Tsirikoglou et al. [601] who present a very realistic effort for the rendering of synthetic data, including Monte Carlobased light transport simulation and simulation of optics and sensors, within the domain of rendering outdoor scenes (see also Section 3.1, where w”
ProcSy: Procedural Synthetic Dataset Generation Towards Influence Factor Studies Of Semantic Segmentation Networks (2019, methodology): “Similar to Synscapes [26], we divide the predicted pixels into subsets of”
AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving (2019, methodology): “Synthetic datasets like Synthia (Ros et al., 2016), Virtual KITTI (Gaidon et al., 2016), Synscapes (Wrenninge and Unger, 2018) offer larger annotated synthetic data for semantic segmentation.”
Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation (2020, background): “Various works have proposed generating synthetic data for tasks such as perception and planning in driving [55,18,53,14,48,68,2], indoor scene perception [70,75,46,57,59,24,4], game playing [6,28], robotic control [61] [6,63,56], optical flow estimation [7,58], home robotics [49,34,20] amongst many ”
Adapting Object Detectors with Conditional Domain Normalization (2020, background): “Synscapes [43] is a synthetic dataset of street scene, which consists of 25, 000 images created with a photo-realistic rendering technique.”
Continuous 3D Perception Model with Persistent State (2025, unknown): “For clarity, we do not differentiate between predictions generated from a raymap or an image in this section to simplify the notation.”
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision (2019, methodology): “All models are trained exclusively on synthetic data (the Virtual KITTI [11] and Synscapes [49] datasets).”
Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation (2020, methodology): “[26] shows, when using the synthetic and real data together, performance increases significantly in t-light and t-sign compared to other classes.”
Unsupervised Intra-Domain Adaptation for Semantic Segmentation Through Self-Supervision (2020, methodology): “Synscapes.”
Meta-Sim: Learning to Generate Synthetic Datasets (2019, background): “The community has been investing significant effort in creating high-quality synthetic content, ranging from driving scenes [38, 11, 36, 8, 34, 47, 2], indoor scenes [48, 50, 33], household robotics [35, 22], robotic control [44], game playing [5], optical flow estimation [6, 23], and quadcopter con”
Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentation (2020, background): “Publications in this direction include the rendering of highly realistic images [35] or using generative adversarial networks (GANs) for generation and augmentation of already collected data [30].”