The ApolloScape Open Dataset for Autonomous Driving and Its Application

By Peng Wang, Xinyu Huang, Xinjing Cheng, Dingfu Zhou, Qichuan Geng, Ruigang Yang. 2018

673 citations recorded by the source corpus.

Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g., KITTI [2] or Cityscapes [3] , ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as joint 3D-2D segment labeling, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “ApolloScape is a very recent dataset that is not fully public yet [278].”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, methodology): “CamVid [8], Cityscapes [19], Mapillary Vistas [33], D(2)-City [11], BDD100k [85] and Apolloscape [41] released ever growing datasets with segmentation masks.”
Vision Transformers for Dense Prediction (2021, methodology): “We construct a meta-dataset that includes the original datasets that were used in [32] (referred to as MIX 5 in that work) and extend it with with five additional datasets ([19, 45, 46, 48, 49]).”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, positive): “Some datasets, such as ApolloScape [119], capture all weather conditions from sunny to snowy.”
A2D2: Audi Autonomous Driving Dataset (2020, background): “Therefore the fact that our semantic segmentation taxonomy has classes car[1-4] does not imply that there is a maximum of 4 cars per image.”
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, methodology): “In contrast to ApolloScape [45] which annotates static objects in 3D and dynamic objects in 2D”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “We consider several key methods and evaluate their performance on the stereo subset of the ApolloScape dataset [34], and on an in-house collected set of four images.”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, methodology): “This method has higher accuracy than GRIP, and at the end of 2019 it ranked ﬁrst in the Baidu Apolloscape dataset [129] ranking.”
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth (2023, methodology): “…we train on a mix of 12 datasets (M12) consisting of the 10 datasets used in [32]: HRWSI [46], Blend-edMVS [47], ReDWeb [45], DIML-Indoor [17], 3D Movies [33], MegaDepth [25], WSVD [41], TartanAir [43], ApolloScape [15] and IRS [42], plus 2 additional datasets: KITTI [29] and NYU Depth v2 [37].”
ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding (2021, background): “Subsequent datasets mostly aimed at increasing the scale [17], diversity [28] and number of tasks [55].”
3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “Appllo [61, 62] 2019 - - - 140K 103 27 Yes Yes Yes Yes Yes Yes 78”
The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset (2019, background): “A number of LIDAR- and vision-based autonomous driving datasets, such as [2]–[9], are available to the community and were primarily collected in order to develop competencies in these modalities.”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, methodology): “Other useful data sets include the Waymo Open [72], Oxford Robotcar [73], ApolloScape [74], Udacity [75], ETH Pedestrian [76], and Caltech Pedestrian [77] data sets.”
DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection (2022, neutral): “…vehicle 15k 15k 200k 8 nuScenes [2] 2019 real single vehicle 1.4M 400k 1.4M 23 Waymo Open [23] 2019 real single vehicle 1M 200k 12M 4 ApolloScape [12] 2018 real single vehicle 144k 0 70k 8-35 BBD100K [30] 2020 real single vehicle 100M 0 0 10 ONCE [17] 2021 real single vehicle 7M 1M 417k 5…”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, neutral): “9k Yes 3 Yes/No Korea thermal images ApolloScape [110, 180] 2019 100 Yes 20k 144k 475k 6 -/- China H3D [214] 2019 0.”
Learning Depth with Convolutional Spatial Propagation Network (2018, neutral): “[19].”