Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning

By Michael Everett, Yu Fan Chen, Jonathan P. How. IEEE/RJS International Conference on Intelligent RObots and Systems, 2018

618 citations recorded by the source corpus.

Robots that navigate among pedestrians use collision avoidance algorithms to enable safe and efficient operation. Recent works present deep reinforcement learning as a framework to model the complex interactions and cooperation. However, they are implemented using key assumptions about other agents' behavior that deviate from reality as the number of agents in the environment increases. This work extends our previous approach to develop an algorithm that learns collision avoidance among a variety of types of dynamic agents without assuming they follow any particular behavior rules. This work also introduces a strategy using LSTM that enables the algorithm to use observations of an arbitrary number of other agents, instead of previous methods that have a fixed observation size. The proposed algorithm outperforms our previous approach in simulation as the number of agents increases, and the algorithm is demonstrated on a fully autonomous robotic vehicle traveling at human walking speed.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f3161b75de1e37b0591f250068b676ea72d1ba22

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep reinforcement learning based mobile robot navigation: A review (2021, methodology): “In 2018, the GA3C-CADRL [69] algorithm combined with LSTM was proposed to avoid collisions between various types of dynamic agents without assuming that they follow any speciﬁc behavioral rules.”
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs (2021, background): “It is also crucial to guarantee safe operation: in many application from selfdriving cars to collaborative robots on factory floors, identifying obstacles is not sufficient for safe and effective navigation/action, and it becomes crucial to capture the dynamic entities in the scene (in particular, h”
Graph Neural Networks for Decentralized Multi-Robot Path Planning (2019, background): “Of particular interest to us is the capability of learningbased methods to handle high-dimensional joint state-space representations, useful when planning for large-scale collective robotic systems, by offloading the online computational burden to an offline learning procedure [20], [21], [19].”
Core Challenges of Social Robot Navigation: A Survey (2021, methodology): “For instance, the majority of recent literature [36, 37, 61, 153] relies on ORCA [262] as the main behavioral engine governing simulated humans for both training and testing of learning algorithms.”
Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios (2020, background): “Chen et al. (2017b), Chen et al. (2017a), and Everett et al. (2018) proposed multi-robot collision-avoid-ance policies by DRL, which requires deploying multiple sensors to estimate the states of nearby agents and moving obstacles.”
3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans (2020, background): “From self-driving cars to collaborative robots on factory floors, identifying obstacles is not sufficient for safe and effective navigation/action, and it becomes crucial to reason on the dynamic entities in the scene (in particular, humans) and predict their behavior or intentions [24].”
Learning a Decentralized Multi-arm Motion Planner (2020, methodology): “While [31] could use a pretrained behavior cloned policy for RL in their sparse reward setting, their application was in path planning for grounded robots in a 2D conﬁguration space which corresponds to the cartesian space, which is signiﬁcantly simpler than motion planning for robotic arms in 6…”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “185) model a Dec-MDP in which the policy takes the state vector consisting of positions, velocities, and radii of all the robots as input to predict the velocities for each robot.”