Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning

By Pinxin Long, Tingxiang Fan, Xinyi Liao, Wenxi Liu, H. Zhang, Jia Pan. 2017

530 citations recorded by the source corpus.

Developing a safe and efficient collision avoidance policy for multiple robots is challenging in the decentralized scenarios where each robot generates its paths without observing other robots' states and intents. While other distributed multi-robot collision avoidance systems exist, they often require extracting agent-level features to plan a local collision-free action, which can be computationally prohibitive and not robust. More importantly, in practice the performance of these methods are much lower than their centralized counterparts. We present a decentralized sensor-level collision avoidance policy for multi-robot systems, which directly maps raw sensor measurements to an agent's steering commands in terms of movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large number of robots on rich, complex environments simultaneously using a policy gradient based reinforcement learning algorithm. We validate the learned sensor-level collision avoidance policy in a variety of simulated scenarios with thorough performance evaluations and show that the final learned policy is able to find time efficient, collision-free paths for a large-scale robot system. We also demonstrate that the learned policy can be well generalized to new scenarios that do not appear in the entire training period, including navigating a heterogeneous group of robots and a large-scale scenario with 100 robots. Videos are available at https://sites.google.com/view/drlmaca.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Liu
2. Qiang Yang
3. Tianjian Chen
4. Kai Zhu
5. Tao Zhang
6. Tingxiang Fan
7. Wenxi Liu
8. Pinxin Long

In Automotive Engineering

1. Kai Zhu
2. Tao Zhang
3. Wenxi Liu
4. Jia Pan
5. Ming Liu
6. Yuejiang Liu
7. Changan Chen
8. Tingxiang Fan

In Computer Vision and Pattern Recognition

1. Kai Zhu
2. Tao Zhang
3. Tingxiang Fan
4. Qiang Yang
5. Pinxin Long
6. Jia Pan
7. Wenxi Liu
8. Yang Liu

Citations in the corpus

• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Long et al. (2017) deal with decentralized multi-agent collision avoidance with PPO.”
• Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios (2020, methodology): “This article is an extension of our previous conference version (Long et al., 2017a), which presents a multi-robot navigation policy that demonstrates the state-of-the-art performance in simulated scenarios.”
• Federated Transfer Reinforcement Learning for Autonomous Driving (2019, background): “s [5]–[7]. With the fast advance of robotics simulators, lots of researches start to investigate the feasibility and effectiveness of transferring the knowledge of simulators to real-life agents [1], [8]–[11]. [8] proposed a decentralized end-to-end sensor-level collision avoidance policy for multi-”
• Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning (2020, background): “In [27], a multi-stage and multi-scenario training framework for training the collision avoidance policy was proposed.”
• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems (2019, methodology): “follow-up works have also been proposed, such as embedding SLAM-like structure into DRL networks [8], or utilizing DRL for multi-robot collision avoidance [9].”
• Deep reinforcement learning based mobile robot navigation: A review (2021, background): “Generally, based on the target distance, moving closer to a target will generate a small positive reward [24, 34, 58, 60] , and moving closer to an obstacle will generate a small negative reward [69] .”
• Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning (2018, methodology): “In the field of robot navigation, recent works have used RL to learn sensorimotor policies in static and dynamic environments from the raw observations [21],”