Imitating driver behavior with generative adversarial networks

By Alex Kuefler, Jeremy Morton, Tim Allan Wheeler, Mykel J. Kochenderfer. 2017 IEEE Intelligent Vehicles Symposium (IV), 2017

448 citations recorded by the source corpus.

The ability to accurately predict and simulate human driving behavior is critical for the development of intelligent transportation systems. Traditional modeling methods have employed simple parametric models and behavioral cloning. This paper adopts a method for overcoming the problem of cascading errors inherent in prior approaches, resulting in realistic behavior that is robust to trajectory perturbations. We extend Generative Adversarial Imitation Learning to the training of recurrent policies, and we demonstrate that our model rivals rule-based controllers and maximum likelihood models in realistic highway simulations. Our model both reproduces emergent behavior of human drivers, such as lane change rate, while maintaining realistic control over long time horizons.

Subfields: Automotive Engineering, Building and Construction, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/ab2e9398954142727aa78ac68517159906e3fffd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Building and Construction

In Control and Systems Engineering

Citations in the corpus

Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction (2017, background): “With the goal of creating robots that interact intelligently with human counterparts, observing data from human-human interactions provides valuable insight into predicting interaction dynamics [1], [2], [3].”
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction (2019, methodology): “crafted cost features, and learn linear weighting functions to rationalize trajectories which are assumed to be generated by optimal control [18]. Recent data-driven approaches based on deep networks [1,4,9,10,13,19,20,24, 28,29,31] outperform traditional approaches. Most of this work focuses either”
Naturalistic Driver Intention and Path Prediction Using Recurrent Neural Networks (2018, background): “Recently, there has been work in this area with a focus on vehicles on highways using processed visual data of surrounding vehicles [12], radar data [13] or simulated Lidar returns of surround vehicles [14].”
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning (2018, background): “Lane merges in SUMO: Previous work modeled autonomous driving tasks as MDPs in which all other vehicles do not learn to respond to a single learning agent [10, 12].”
Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs (2018, methodology): “We compare our trajectory prediction results with those reported in [4].”
InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations (2017, methodology): “Most imitation learning methods for end-to-end driving rely heavily on LIDAR-like inputs to obtain precise distance measurements [21, 41].”
Learning From Demonstration in the Wild (2018, background): “Most relevant is learning highway merging behaviour [33], [34] from NGSIM [35], a publicly available dataset of vehicle trajectories.”
Human motion trajectory prediction: a survey (2019, methodology): “This method has been successfully applied to learning human highway driving behavior (Kueﬂer et al. 2017) and training joint pedestrian motion models (Gupta et al. 2018).”
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst (2018, background): “But practically, we needed ways to address this challenge without exposing demonstrators to new states actively (Ross et al. (2011); Laskey et al. (2017b)) or performing reinforcement learning (Kueﬂer et al. (2017)).”
Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving (2019, methodology): “been established with two approaches: rule-based and supervised learning [3], [4].”
Convolutional Social Pooling for Vehicle Trajectory Prediction (2018, methodology): “Evaluation Metric Prediction horizon (s) CV C-VGMM + VIM [6] GAIL-GRU [13] V-LSTM S-LSTM CS-LSTM CS-LSTM(M)”
A Survey of State-Action Representations for Autonomous Driving (2018, methodology): “The representation choice can be di(cid:27)erent for dynamic and static obstacles, as shown by Pfei(cid:27)er et al. (2017a) who use a Cartesian grid to encode the static environment and a polar grid for the location of pedestrians.”
Multiple Futures Prediction (2019, unknown): “CVGMM[15] [23] MATF[39] LSTM S-LSTM[1] CS-LSTM(M) MFP-1 MFP-2 MFP-3 MFP-4 MFP-5”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, background): “[182] extend GAIL to the optimization of RNN to demonstrate human driver behaviors, and policies and actions are evaluated by the discriminator.”
Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations (2018, methodology): “In addition, RL maximizes the overall expected return on a full trajectory, while IL treats every observation independently [6], which con-”
GRIP: Graph-based Interaction-aware Trajectory Prediction (2019, methodology): “Prediction Horizon (s) CV V-LSTM C-VGMM + VIM [10] GAIL-GRU [15] CS-LSTM(M) [11] CS-LSTM [11] GRIP (4CS-LSTM) GRIP (ALL) 1 0.73 0.68 0.66 0.69 0.62 0.61 0.37 (40%↑ -0.24) 0.64 2 1.78 1.65 1.56 1.51 1.29 1.27 0.86 (32%↑ -0.41) 1.13 3 3.13 2.91 2.75 2.55 2.13 2.09 1.45 (31%↑ -0.64) 1.80 4 4.78 4.46 4.”
A Traffic Flow Simulation Framework for Learning Driver Heterogeneity from Naturalistic Driving Data using Autoencoders (2019, background): “This progress led the development of machine-learning based models for traffic simulations (14)-(16) and driving behavior analysis (17)-(18) .”
Planning and Decision-Making for Autonomous Vehicles (2018, background): “(124) demonstrated the effectiveness of generative adversarial imitation learning (125), extended to the optimization of recurrent policies.”
Robust Imitation of Diverse Behaviors (2017, methodology): “rt-like regions of the state-action space. In practice, trust region policy optimization (TRPO) is used to stabilize the learning process [30]. GAIL has become a popular choice for imitation learning [16] and there already exist model-based [3] and third-person [35] extensions. Two recent GAIL-based”