TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

By Simon Suo, S. Regalado, Sergio Casas, Raquel Urtasun. Computer Vision and Pattern Recognition, 2021

321 citations recorded by the source corpus.

Simulation has the potential to massively scale evaluation of self-driving systems, enabling rapid development as well as safe deployment. Bridging the gap between simulation and the real world requires realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration, and thus capture more naturalistic driving behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we parameterize the policy with an implicit la-tent variable model that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.

Subfields: Automotive Engineering, Building and Construction, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/9868cbb66312c7790602d4e3fe44af1482d5ce54

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Building and Construction

In Control and Systems Engineering

Citations in the corpus

3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, background): “There are also some works simulating the traffic flows (Tan et al., 2021; Suo et al., 2021) or testing the safety of vehicles by simulation (Wong et al.”
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach (2021, background): “For urban driving simulators, the realistic behavior of road users is of utmost importance [44].”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, methodology): “Evaluation of interactive prediction models requires metrics formulated for joint predictions as motivated by recent work [32, 6, 33, 28].”
V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception (2023, background): “However, it is well known that there exists a clear domain gap between synthetic data and real-world data, as the traffic behavior and sensor rendering in simulators are often not realistic enough [27, 37].”
DriveLM: Driving with Graph Visual Question Answering (2023, methodology): “For example, Traf-ficSim [64] collects more data for safety-critical cases by simulation.”
UniSim: A Neural Closed-Loop Sensor Simulator (2023, background): “While we only demonstrated human-specified trajectories in the paper, it is intuitive to to incorporate other models to control actors’ behaviors, such as intelligent driver models [34,78] or deep-learning based models [29,73].”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, unknown): “Awidevarietyof learning-basedtechniqueshavebeenproposedforthistask[97], [98],[100],[102],[103],[104].”
Parting with Misconceptions about Learning-based Vehicle Motion Planning (2023, background): “nuPlan constructs a simulated environment as closely as possible to a real-world driving setting through data-driven simulation [53, 54, 55, 56, 57, 58, 59].”