Scalability in Perception for Autonomous Driving: Waymo Open Dataset
By Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, P. Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine and 12 more. 2019
4,247 citations recorded by the source corpus.
The research community has increasing interest in autonomous driving research, despite the resource intensity of obtaining representative real world data. Existing self-driving datasets are limited in the scale and variation of the environments they capture, even though generalization within and between operating regions is crucial to the over-all viability of the technology. In an effort to help align the research community’s contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset. Our new dataset consists of 1150 scenes that each span 20 seconds, consisting of well synchronized and calibrated high quality LiDAR and camera data captured across a range of urban and suburban geographies. It is 15x more diverse than the largest camera+LiDAR dataset available based on our proposed diversity metric. We exhaustively annotated this data with 2D (camera image) and 3D (LiDAR) bounding boxes, with consistent identifiers across frames. Finally, we provide strong baselines for 2D as well as 3D detection and tracking tasks. We further study the effects of dataset size and generalization across geographies on 3D detection methods. Find data, code and more up-to-date information at http://www.waymo.com/open.
Subfields: Aerospace Engineering, Automotive Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Andreas Geiger
- Xiaogang Wang
- Sergey Levine
- Jitendra Malik
- Marc Pollefeys
- Hongsheng Li
- Silvio Savarese
- Luc Van Gool
In Automotive Engineering
- Andreas Geiger
- Kashyap Chitta
- Kris M. Kitani
- Hongyang Li
- Benjamin Sapp
- Li Chen
- Shaoshuai Shi
- Peng Wu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Jitendra Malik
- Alan L. Yuille
- Andrew Zisserman
Citations in the corpus
- BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning (2018, neutral): “A recent dataset released by Waymo [29] has fewer tracking sequences (1150 vs 2000) and fewer frames (230K vs 398K) in total, compared to ours.”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…trained models can suffer at segmenting nighttime driving scenes (Dai and Van Gool, 2018), detecting relevant objects in new or challenging locations and settings (Yu et al., 2020; Sun et al., 2020), and, as discussed earlier, detecting pedestrians with darker skin tones (Wilson et al., 2019).”
- Robust fine-tuning of zero-shot models (2021, background): “In contrast, accuracy on the reference distribution is often a reliable predictor for accuracy under distribution shift [67, 68, 92, 95, 104].”
- VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (2023, methodology): “VoxelNeXt presents promising results on large-scale datasets, including nuScenes [3], Waymo [45], and Argoverse2 [52].”
- A2D2: Audi Autonomous Driving Dataset (2020, methodology): “Most recently, the Waymo Open Dataset [4] was released with 12 million 3D bounding box annotations on LiDAR point clouds and 1.”
- LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving (2020, background): “Although several large datasets have been recently published by academia [5] and autonomous driving companies [6], [7], [8], these datasets which are collected from real world drives are usually manually (often with help from some automated tools) labeled, which is slow, costly, and error-prone.”
- Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, positive): “We select three real-world videos datasets: RE10K [74], DL3DV [29], Waymo Open Dataset (WOD) [49], and a synthetic dataset Kubric4D [14, 53].”
- 3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “, nuScenes [37], and Waymo Open [7]), algorithms (e.”
- PointAugmenting: Cross-Modal Augmentation for 3D Object Detection (2021, methodology): “Waymo Open Dataset [18] is currently the largest dataset for autonomous driving.”
- ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding (2021, background): “…are recent efforts to include adverse visual domains in large-scale datasets, such as Oxford RobotCar [27] and BDD100K [55], these efforts focus either on localization/mapping tasks [27,49] or on recognition tasks which do not involve dense pixel-level outputs , such as object detection [3,42,55].”
- Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, background): “We also conduct experiments on the larger Waymo Open Dataset to further validate the effectiveness of our proposed Voxel R-CNN.”
- Simple Training Strategies and Model Scaling for Object Detection (2021, positive): “Results: We evaluate our improved baselines with SpineNet143L as the backbone on the Waymo Open Dataset (WOD) 2D detection task [32].”
- Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation (2020, background): “ring, for instance, 90 minutes per image [17] or 22 worker hours per 1,000 mask segmentations [49]. In the case of self-driving cars, the annotation of video is a critical supervised learning problem [24,75], and in turn has fostered an industry of specialized companies for data annotation. In contr”
- Octo: An Open-Source Generalist Robot Policy (2024, background): “There are multiple large robot navigation and autonomous driving datasets [29, 95, 13, 87, 80, 43, 89].”
- HOTA: A Higher Order Metric for Evaluating Multi-object Tracking (2020, background): “ing boxes (Sun et al. 2019; Chang et al. 2019), segmentation masks (Voigtlaender et al.”