Planning for Autonomous Cars that Leverage Effects on Human Actions

By Dorsa Sadigh, S. Sastry, Sanjit A. Seshia, Anca Dragan. 2016

557 citations recorded by the source corpus.

—Traditionally, autonomous cars make predictions about other drivers’ future trajectories, and plan to stay out of their way. This tends to result in defensive and opaque behaviors. Our key insight is that an autonomous car’s actions will actually affect what other cars will do in response, whether the car is aware of it or not. Our thesis is that we can leverage these responses to plan more efﬁcient and communicative behaviors. We model the interaction between an autonomous car and a human driver as a dynamical system, in which the robot’s actions have immediate consequences on the state of the car, but also on human actions. We model these consequences by approximating the human as an optimal planner, with a reward function that we acquire through Inverse Reinforcement Learning. When the robot plans with this reward function in this dynamical system, it comes up with actions that purposefully change human state: it merges in front of a human to get them to slow down or to reach its own goal faster; it blocks two lanes to get them to switch to a third lane; or it backs up slightly at an intersection to get them to proceed ﬁrst. Such behaviors arise from the optimization, without relying on hand-coded signaling strategies and without ever explicitly modeling communication. Our user study results suggest that the robot is indeed capable of eliciting desired changes in human state by planning using this dynamical system.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Tour of Reinforcement Learning: The View from Continuous Control (2018, background): “Humans can be modeled as solving their own optimal control problem, and then the human’s actions enter as a disturbance in the optimal control problem [62].”
Shared autonomy via hindsight optimization for teleoperation and teaming (2017, background): “Similarly, Sadigh et al. (2016b) generate motions for an autonomous car using predictions of how other drivers will respond, enabling them to change the behavior of other users, and infer the internal user state (Sadigh et al., 2016a).”
Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction (2018, background): “[26] propose using an autonomous vehicle’s actions to communicate awareness and intent to drivers of manually-driven vehicles.”
Risk-sensitive Inverse Reinforcement Learning via Coherent Risk Models (2017, methodology): “Inverse Reinforcement Learning (IRL) [41, 32, 2, 29, 38, 50, 16] is a powerful and flexible framework for tackling these challenges and has been previously used for tasks such as modeling and mimicking human driver behavior [1, 28, 43], pedestrian trajectory prediction [51, 31], and legged robot loc”
Imitating driver behavior with generative adversarial networks (2017, background): “Because of these benefits, some recent efforts in human driver modeling emphasize IRL [14], [15].”
Flow: A Modular Learning Framework for Mixed Autonomy Traffic (2017, background): “Low-level vehicle dynamics, however, are crucial [61] because many low-level traffic phenomena affect energy consumption, safety, and travel time [14], [62], [63], [64].”
Towards Verified Artificial Intelligence (2016, background): “Initial work by the authors [57, 60, 59] has shown how data gathered from driving simulators via human subject experiments can be used to generate models of human driver behavior that are useful for verification and control of autonomous vehicles.”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, background): “Feature-based IRL [80] teaches the different driving styles in the highway scenario.”
Learning with Humans in the Loop (2020, methodology): “ON THE UTILITY OF MODEL LEARNING IN HUMAN-ROBOT INTERACTION 8 reinforcement learning, and the ToM-based method will plan the robot’s actions using the learned model.”
A Survey of State-Action Representations for Autonomous Driving (2018, methodology): “Sadigh et al. (2016) and Huang et al. (2017) show that this approach can be used in pair with a dynamics model to predict future human actions, which yields a closed loop predictive model.”
Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control (2017, background): “Low-level vehicle dynamics, however, are of crucial importance, as exhibited by [24] and because many trafﬁc phenomena, which affect energy consumption, safety, and travel time are exhibited at the level of low-level dynamics [5], [25], [26], [27].”
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction (2017, methodology): “This framework is employed in the context of interactive driving in [18] to construct robot policies that avoid being overly defensive by leveraging expected human responses to robot actions.”
On the Utility of Learning about Humans for Human-AI Coordination (2019, background): “Much work focuses on achieving collaboration by planning and learning with (non-optimal) models of human behavior [26, 21, 31], as well as on specific properties of robot behavior that aid collaboration [2, 14, 9].”
Convergence of Learning Dynamics in Stackelberg Games (2019, background): “Examples include human-robot collaboration and interacting autonomous systems in artificial intelligence [14, 26, 33, 38], incentive design and control [13, 35, 36], and organizational structures in economics [2, 10].”