Hierarchical Multiagent Reinforcement Learning for Maritime Traffic Management

By Arambam James Singh, Akshat Kumar, Hoong Chuin Lau. 2020

41 citations recorded by the source corpus.

Increasing global maritime traffic coupled with rapid digitization and automation in shipping mandate developing next generation maritime traffic management systems to mitigate congestion, increase safety of navigation, and avoid collisions in busy and geographically constrained ports (such as Singapore's). To achieve these objectives, we model the maritime traffic as a large multiagent system with individual vessels as agents, and VTS (Vessel Traffic Service) authority as a regulatory agent. We develop a hierarchical reinforcement learning approach where vessels first select a high level action based on the underlying traffic flow, and then select the low level action that determines their future speed. We exploit the nature of collective interactions among agents to develop a policy gradient approach that can scale up to large real world problems. We also develop an effective multiagent credit assignment scheme that significantly improves the convergence of policy gradient. Extensive empirical results on synthetic and real world data from one of the busiest port in the world show that our approach consistently performs significantly better than the previous best approach.

Subfields: Artificial Intelligence, Automotive Engineering, Civil and Structural Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tonghan Wang
2. Chongjie Zhang
3. Chenghao Li
4. Qianchuan Zhao
5. Jun Yang
6. Yaodong Yang
7. Jun Wang
8. Chengjie Wu

Citations in the corpus

• Celebrating Diversity in Shared Multi-Agent Reinforcement Learning (2021, background): “Cooperative multi-agent reinforcement learning (MARL) has drawn increasing interest in recent years, which provides a promise for solving many real-world challenging problems, such as sensor networks [1], traffic management [2], and coordination of robot swarms [3].”
• A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “transportation schedule [22, 53, 159, 253], traffic signal control [50, 59], energy management [180], wireless security [172], satellite docking [76], edge computing [299], chemical processes [241], video games [28, 153, 192, 211, 262, 300], board games of Go, shogi, chess and arcade game PAC-MAN [1”