PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning

By Guillaume Sartoretti, Justin Kerr, Yunfei Shi, Glenn Wagner, T. K. Satish Kumar, Sven Koenig, Howie Choset. IEEE Robotics and Automation Letters, 2018

417 citations recorded by the source corpus.

Multi-agent path finding (MAPF) is an essential component of many large-scale, real-world robot deployments, from aerial swarms to warehouse automation. However, despite the community's continued efforts, most state-of-the-art MAPF planners still rely on centralized planning and scale poorly past a few hundred agents. Such planning approaches are maladapted to real-world deployments, where noise and uncertainty often require paths be recomputed online, which is impossible when planning times are in seconds to minutes. We present PRIMAL, a novel framework for MAPF that combines reinforcement and imitation learning to teach fully decentralized policies, where agents reactively plan paths online in a partially observable world while exhibiting implicit coordination. This framework extends our previous work on distributed learning of collaborative policies by introducing demonstrations of an expert MAPF planner during training, as well as careful reward shaping and environment sampling. Once learned, the resulting policy can be copied onto any number of agents and naturally scales to different team sizes and world dimensions. We present results on randomized worlds with up to 1024 agents and compare success rates against state-of-the-art MAPF planners. Finally, we experimentally validate the learned policies in a hybrid simulation of a factory mockup, involving both real world and simulated robots.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Mechanical Engineering

View Paper: https://www.semanticscholar.org/paper/4fe9142a47b35638edcc59013ad5e9e87bd93ea7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Mechanical Engineering

Citations in the corpus

Quo vadis artificial intelligence? (2022, background): “In industrial assembly processes, smart robots or those known as collaborative robots help human operators (and vice versa) to manipulate tools and give warnings and suggestions [102, 103].”
Graph Neural Networks for Decentralized Multi-Robot Path Planning (2019, methodology): “, as implemented in [7]) is harder to train, and is left for future work.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “Finally, Sartoretti et al. (189) used DRL to prevent agents from blocking each other in multi-agent pathfinding problems.”
Mobile Robot Path Planning in Dynamic Environments Through Globally Guided Reinforcement Learning (2020, methodology): “Other approaches apply Imitation Learning (IL) to provide the robot dense rewards to relieve this issue [19]–[21].”
Deep reinforcement learning based mobile robot navigation: A review (2021, background): “[65] PRIMAL (IL + A3C + LSTM) E Dis, 5 Dense Y Y 2019”