Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

By S. Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Benjamin Sapp, Charles Ruizhongtai Qi, Yin Zhou and 8 more. IEEE International Conference on Computer Vision, 2021

896 citations recorded by the source corpus.

As autonomous driving systems mature, motion forecasting has received increasing attention as a critical requirement for planning. Of particular importance are interactive situations such as merges, unprotected turns, etc., where predicting individual object motion is not sufficient. Joint predictions of multiple objects are required for effective route planning. There has been a critical need for high-quality motion data that is rich in both interactions and annotation to develop motion planning models. In this work, we introduce the most diverse interactive motion dataset to our knowledge, and provide specific labels for interacting objects suitable for developing joint prediction models. With over 100,000 scenes, each 20 seconds long at 10 Hz, our new dataset contains more than 570 hours of unique data over 1750 km of roadways. It was collected by mining for interesting interactions between vehicles, pedestrians, and cyclists across six cities within the United States. We use a high-accuracy 3D auto-labeling system to generate high quality 3D bounding boxes for each road agent, and provide corresponding high definition 3D maps for each scene. Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models. Finally, we provide strong baseline models for individual-agent prediction and joint-prediction. We hope that this new large-scale interactive motion dataset will provide new opportunities for advancing motion forecasting models.

Subfields: Automotive Engineering, Building and Construction, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/b07fb2dbaeda171c5f1d821f492dd0cb8bd4e668

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

1. Yu-Kai Huang
2. Jianping Wang
3. Zikang Zhou
4. Yung-Hui Li
5. Shaoshuai Shi
6. Bernt Schiele
7. Dengxin Dai
8. Li Jiang

In Building and Construction

1. Juraj Kabzan
2. Yu-Kai Huang
3. Jianping Wang
4. Zikang Zhou
5. Yung-Hui Li
6. Bernt Schiele

In Safety, Risk, Reliability and Quality

1. Alex Kendall

Citations in the corpus

• Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting (2023, methodology): “Each scenario carries its own local map region, similar to the Waymo Open Motion [12] dataset.”
• Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, methodology): “We evaluate our approach on the large-scale Waymo Open Motion Dataset (WOMD) [14], which mines interesting interactions from real-world traffic scenes and is currently the most diverse interactive motion dataset.”
• MotionDiffuser: Controllable Multi-Agent Motion Prediction Using Diffusion (2023, methodology): “In Table 1, we report the main metrics for the Interactive split, as defined in [7].”
• Query-Centric Trajectory Prediction (2023, background): “Please refer to the supplementary material for more results on Argoverse 2 [49] and Waymo Open Motion Dataset [13].”
• nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles (2021, background): “Existing real-world benchmarks are focused on shortterm motion forecasting, also known as prediction [6, 4, 11, 8], rather than planning.”
• MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, methodology): “This is a criterion that explicitly optimizes trajectory diversity, which is a fit for metrics such as miss rate, mAP and minADE, as defined in [12, 18].”
• Planning-oriented Autonomous Driving (2022, background): “And the bounding boxes are ground truth annotations in most current motion datasets [27], which is not realistic in onboard scenarios.”
• Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys (2023, neutral): “He was the recipient many awards and honors, selectively including the Highly Commended Paper Award of IMechE U.K. in 2012, Japan NSK Outstanding Mechanical Engineering Paper Award in 2014, Tsinghua University Outstanding Doctoral Thesis Award in 2016, IEEE IV Best Workshop/Special Session Paper Awa”
• GAIA-1: A Generative World Model for Autonomous Driving (2023, background): “To achieve this, the development of a robust model of the world is imperative [1] and huge efforts have been made in the past to build such predictive world models for autonomous driving [2, 3, 4, 5, 6].”