The ApolloScape Dataset for Autonomous Driving

By Xinyu Huang, Xinjing Cheng, Qichuan Geng, Binbin Cao, Dingfu Zhou, Peng Wang, Yuanqing Lin, Ruigang Yang. 2018

677 citations recorded by the source corpus.

Scene parsing aims to assign a class (semantic) label for each pixel in an image. It is a comprehensive analysis of an image. Given the rise of autonomous driving, pixel-accurate environmental perception is expected to be a key enabling technical piece. However, providing a large scale dataset for the design and evaluation of scene parsing algorithms, in particular for outdoor scenes, has been difficult. The per-pixel labelling process is prohibitively expensive, limiting the scale of existing ones. In this paper, we present a large-scale open dataset, ApolloScape, that consists of RGB videos and corresponding dense 3D point clouds. Comparing with existing datasets, our dataset has the following unique properties. The first is its scale, our initial release contains over 140K images - each with its per-pixel semantic mask, up to 1M is scheduled. The second is its complexity. Captured in various traffic conditions, the number of moving objects averages from tens to over one hundred (Figure 1). And the third is the 3D attribute, each image is tagged with high-accuracy pose information at cm accuracy and the static background point cloud has mm relative accuracy. We are able to label these many images by an interactive and efficient labelling pipeline that utilizes the high-quality 3D point cloud. Moreover, our dataset also contains different lane markings based on the lane colors and styles. We expect our new dataset can deeply benefit various autonomous driving related applications that include but not limited to 2D/3D scene understanding, localization, transfer learning, and driving simulation.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, background): “with multiple annotations and tasks, towards learning intermediate representations through downstream and multi-task learning on the same input. This has been recently achieved for autonomous driving [17,18,19,20,21,22,23,24] and scene understanding [25,26]. For example, Taskonomy [25] contains 26 t”
WoodScape: A Multi-Task, Multi-Camera Fisheye Dataset for Autonomous Driving (2019, background): “Task/Info Quantity KITTI [17] Cityscapes [10] Mapillary [39] nuScenes [6] ApolloScape [24] BDD100k [59] WoodScape Ours”
IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, methodology): “Apollo Stereo [18], like KITTI, is also a dataset for autonomous driving, but it uses two VUX-1HAs to get more denser depth information, providing a total of 22,390 pairs for training.”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Cityscapes [270] X X Vehicle Berkley DeepDrive [271] X X Vehicle X X Mapillary [272] X X Vehicle X X Oxford RobotCar [49] X X Vehicle X X X KITTI [174] X X X X Vehicle H3D [273] X X X Vehicle ApolloScape [274] X X X X Vehicle nuScenes [175] X X X X Vehicle X X Udacity [275] X X X X Vehicle DDD17 [85”
Video-to-Video Synthesis (2018, background): “• Apolloscape [29] consists of 73 street scene videos captured in Beijing, whose video lengths vary from 100 to 1000 frames.”
MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, background): “Excluded Datasets APOLLOSCAPE [56] Driving (Worldwide) 147,000 We include only fully-manually labeled datasets in MSeg.”
The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes (2019, background): “The field has been significantly boosted by recent advances of deep learning algorithms [2], [3], [4], [5], [6] and an increasing number of datasets [7], [8], [9], [10], [11].”
D2-City: A Large-Scale Dashcam Video Dataset of Diverse Traffic Scenarios (2019, methodology): “Similar to ApolloScape [10], we paid special attention to three-wheel vehicles which are common on roads in China.”
Deep Evidential Regression (2019, methodology): “In this subsection, we investigate the ability of evidential models to capture increased epistemic uncertainty on OOD data, by testing on images from ApolloScape [21], an OOD dataset of diverse outdoor driving.”
Segment as Points for Efficient Online Multi-Object Tracking and Segmentation (2020, methodology): “Moreover, to facilitate better evaluations, we construct a more crowded thus more challenging MOTS dataset named APOLLO MOTS based on the public ApolloScape dataset [13].”
Complex urban dataset with multi-level sensors from highly diverse urban environments (2019, methodology): “Whereas the aforementioned datasets focus on a single sensor modality, multi-sensor datasets (Huang et al., 2010, 2018; Maddern et al., 2017) provide data related to cameras, LiDAR, and navigation sensors to enable users to select data according to the purpose in hand.”
Few-shot Video-to-Video Synthesis (2019, methodology): “At test time, in addition to the test set images from these three areas, we also test on the ApolloScape [20] and CamVid [5] datasets, which are not included in the training set.”
MOTS: Multi-Object Tracking and Segmentation (2019, background): “Cityscapes [12], BDD [62], and ApolloScape [19] provide video data for an automotive scenario.”
Urban Radiance Fields (2021, background): “Neither system leverages lidar data, which is available in most outdoor mapping platforms [6, 7, 20, 22, 25, 32, 36, 61, 63] nor do they attempt to extract 3D surface reconstructions.”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, background): “Despite many explored ideas, the current most popular datasets for object instances localization were all obtained using purely manual annotations of boxes or (layered) polygons [8, 31, 9, 19, 6, 24, 11, 37].”
Cooper: Cooperative Perception for Connected Autonomous Vehicles Based on 3D Point Clouds (2019, background): “Rapid development of autonomous vehicles has motivated research institutions to develop representations to perceive local environment, such as lane detection, traffic sign detection and detect objects like cars, cyclists and pedestrians [19], [20], [29], [31] based on the open datasets [7], [9], [10”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, methodology): “Other useful data sets include the Waymo Open [72], Oxford Robotcar [73], ApolloScape [74], Udacity [75], ETH Pedestrian [76], and Caltech Pedestrian”