Socially aware motion planning with deep reinforcement learning

By Yu Fan Chen, Michael Everett, Miao Liu, Jonathan P. How. 2017

777 citations recorded by the source corpus.

For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e.g., passing on the right). However, while instinctive to humans, socially compliant navigation is still difficult to quantify due to the stochasticity in people's behaviors. Existing works are mostly focused on using feature-matching techniques to describe and imitate human paths, but often do not generalize well since the feature values can vary from person to person, and even run to run. This work notes that while it is challenging to directly specify the details of what to do (precise mechanisms of human navigation), it is straightforward to specify what not to do (violations of social norms). Specifically, using deep reinforcement learning, this work develops a time-efficient navigation policy that respects common social norms. The proposed method is shown to enable fully autonomous navigation of a robotic vehicle moving at human walking speed in an environment with many pedestrians.

Subfields: Artificial Intelligence, Automotive Engineering, Ocean Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Ocean Engineering

In Robotics & Control

Citations in the corpus

Unmasking Clever Hans predictors and assessing what machines really learn (2019, background): “social norms which penalize such behavior [35]) are incorporated into the evaluation metric.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “They extend a prior work (Chen et al., 2017b), and build a robotic system for their real-world experiment, where a differential-drive mobile robot is mounted with a Lidar for localization and three Intel Realsense ’s for obstacle avoidance.”
Stochastic Primal-Dual Q-Learning (2018, neutral): “The problem of learning a map from world observations to actions, called a policy, lies at the core of many sequential decision problems, such as robotics (Chen et al., 2017), artiﬁcial intelligence (Mnih et al., 2015), ﬁnance (Longstaﬀ and Schwartz, 2001), and economics (Tesauro and Kephart, 2002).”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, background): “Much of the reinforcement learning work for robot navigation has focused on collision avoidance [27, 28, 15], while our work addresses goal-directed, multi-objective navigation.”
Deep reinforcement learning based mobile robot navigation: A review (2021, background): “In 2017, Chen et al. [67] extended their previous work CADRL to social navigation scenarios, and proposed the socially aware CADRL algorithm.”
Core Challenges of Social Robot Navigation: A Survey (2021, methodology): “[37, 39], there is no reason to expect this complexity issue to disappear; rather, we expect that the complexity will be reflected in training burden.”
Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning (2017, methodology): “This limitation restricts these methods to be only applicable for robots equipped with high precision sensors, like 3D Lidars [2], [3].”
Stochastic Primal-Dual Q-Learning Algorithm For Discounted MDPs (2019, background): “The problem of learning a map from world observations to actions, called a policy, lies at the core of many sequential decision problems, such as robotics [1], artificial intelligence [2], finance [3], and economics [4].”
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning (2017, methodology): “robot motion planning, they have mainly focused on a discrete action space [13], [21] or on small-scale problems [8], [9], [16], [17].”
Deep Local Trajectory Replanning and Control for Robot Navigation (2019, neutral): “Generally, most methods for social navigation explicitly model interactions among agents or social conventions [5, 31, 7, 32, 33, 34, 35, 36, 37, 38, 39].”
A Survey of State-Action Representations for Autonomous Driving (2018, background): “Higher-level representations can also be used if we ignore some parts of the dynamics, as showed by Chen et al. (2017) who act directly with longitudinal and lateral velocities.”
Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning (2018, background): “[36] and socially cooperative policies with the agent-level state information [19], [20], [22].”
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning (2018, methodology): “We use a local coordinate frame as in [2], [3], and separate the state of the world in two pieces: information about the agent itself, and everything else in the world.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “Third, when human behaviors are simple, human models can be directly hardcoded using domain knowledge (175, 176, 177), and be incorporated either as parts of the simulation or as behavioral constraints.”
Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning (2020, background): “Previous works pioneered the use of deep reinforcement learning in navigation around pedestrians and developed multi-agent collision avoidance systems focused on optimizing path efficiency (CADRL)[23] and using the reward function to induce social norms such as passing on the right (SA-CADRL)[24].”
Planning and Decision-Making for Autonomous Vehicles (2018, background): “socially aware behaviors capable of learning directly from multiagent scenarios by developing a symmetrical neural network structure (79).”
Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios (2020, methodology): “In this way, the sensor-level collision avoidance is more generalized and robust than the agent-level collision-avoid-ance formulation in Chen et al. (2017a,b), van den Berg et al. (2008), and van den Berg et al. (2011a).”
Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations (2018, background): “[26] presented a RL approach for collision avoidance in dynamic environments.”