Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning

By Yu Fan Chen, Miao Liu, Michael Everett, Jonathan P. How. 2016

697 citations recorded by the source corpus.

Finding feasible, collision-free paths for multiagent systems can be challenging, particularly in non-communicating scenarios where each agent's intent (e.g. goal) is unobservable to the others. In particular, finding time efficient paths often requires anticipating interaction with neighboring agents, the process of which can be computationally prohibitive. This work presents a decentralized multiagent collision avoidance algorithm based on a novel application of deep reinforcement learning, which effectively offloads the online computation (for predicting interaction patterns) to an offline learning procedure. Specifically, the proposed approach develops a value network that encodes the estimated time to the goal given an agent's joint configuration (positions and velocities) with its neighbors. Use of the value network not only admits efficient (i.e., real-time implementable) queries for finding a collision-free velocity vector, but also considers the uncertainty in the other agents' motion. Simulation results show more than 26% improvement in paths quality (i.e., time to reach the goal) when compared with optimal reciprocal collision avoidance (ORCA), a state-of-the-art collision avoidance strategy.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “They extend a prior work (Chen et al., 2017b), and build a robotic system for their real-world experiment, where a differential-drive mobile robot is mounted with a Lidar for localization and three Intel Realsense ’s for obstacle avoidance.”
(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image (2016, background): “Several recent works have extended deep RL methods to real-world robotics applications using either lowdimensional estimated state [10] or by collecting an exhaustive real-world dataset under gridworld-like assumptions [43].”
Socially aware motion planning with deep reinforcement learning (2017, methodology): “The CADRL work [14] trained a two-agent network with three fully connected hidden layers, and used a minimax scheme for scaling up to multiagent ( n > 2 ) scenarios.”
Deep reinforcement learning based mobile robot navigation: A review (2021, methodology): “Generally, based on the target distance, moving closer to a target will generate a small positive reward [24, 34, 58, 60] , and moving closer to an obstacle will generate a small negative reward [69] .”
Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning (2018, methodology): “We follow the robot-centric parameterization in [19], [22], where the robot is located at the origin and the x-axis is pointing toward the robot’s goal.”
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning (2017, methodology): “robot motion planning, they have mainly focused on a discrete action space [13], [21] or on small-scale problems [8], [9], [16], [17].”
Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios (2020, methodology): “Although DRL algorithms have been successfully applied to mobile robot motion planning, they have mainly focused on a discrete action space (Zhang et al., 2016a; Zhu et al., 2017) or on small-scale problems (Chen et al., 2017a,b; Kahn et al., 2017; Tai et al., 2017).”
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning (2018, methodology): “We use a local coordinate frame as in [2], [3], and separate the state of the world in two pieces: information about the agent itself, and everything else in the world.”
Federated Transfer Reinforcement Learning for Autonomous Driving (2019, background): “[1] studied the problem of reducing the computationally prohibitive process of anticipating interaction with neighboring agents in a decentralized multi-agent collision avoidance scenario.”
Core Challenges of Social Robot Navigation: A Survey (2021, methodology): “For instance, Chen et al. [32], Fan et al. [55] employ CADRL [33], a decentralized deep reinforcement learning framework for socially aware multiagent collision avoidance.”
A review of cooperative multi-agent deep reinforcement learning (2019, background): “approaches to path adjustments is “velocity planning” [36, 42], which modifies the velocity profile of each agent along with its path to avoid collisions.”
PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning (2018, background): “Another common approach is to rely on reactive planners, which do not plan joint paths for all agents before execution, but rather correct individual paths online to avoid collisions [5], [10].”
Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning (2020, background): “ed deep RL to navigation and developed multi-agent collision avoidance systems focused on optimizing path efﬁciency while ﬁnding a feasible trajectory around a potentially large number of pedestrians [23,24,8]. This line of work showed that is possible to understand and generate dynamic navigation b”
Human motion trajectory prediction: a survey (2019, methodology): “Map-aware methods , which account for environment geometry and topology, e.g. (Ziebart et al. 2009; Vasquez 2016; Pfeiffer et al. 2016; Chen et al. 2017; Pool et al. 2017; Rudenko et al. 2017, 2018b; Kooij et al. 2018; Henry et al. 2010; Ikeda et al. 2012; Liao et al. 2003; Chung and Huang 2010;…”