Deep Multi-Agent Reinforcement Learning with Relevance Graphs

By Aleksandra Malysheva, Tegg Taekyong Sung, Chae-Bong Sohn, Daniel Kudenko, Aleksei Shpilman. 2018

50 citations recorded by the source corpus.

Over recent years, deep reinforcement learning has shown strong successes in complex single-agent tasks, and more recently this approach has also been applied to multi-agent domains. In this paper, we propose a novel approach, called MAGnet, to multi-agent reinforcement learning (MARL) that utilizes a relevance graph representation of the environment obtained by a self-attention mechanism, and a message-generation technique inspired by the NerveNet architecture. We applied our MAGnet approach to the Pommerman game and the results show that it significantly outperforms state-of-the-art MARL solutions, including DQN, MADDPG, and MCTS.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shimon Whiteson
2. Christian Schröder de Witt
3. Tabish Rashid
4. Zongqing Lu
5. Gregory Farquhar
6. Jakob N. Foerster
7. Chongjie Zhang
8. Mikayel Samvelyan

In Control and Systems Engineering

1. Shimon Whiteson
2. Jakob N. Foerster
3. Zongqing Lu
4. Tabish Rashid
5. Jiechuan Jiang
6. Christian Schröder de Witt
7. Mikayel Samvelyan

In Reinforcement Learning

1. Shimon Whiteson
2. Christian Schröder de Witt
3. Tabish Rashid
4. Zongqing Lu
5. Gregory Farquhar
6. Jakob N. Foerster
7. Chongjie Zhang
8. Jiechuan Jiang

In Graph Learning

1. Christian Schröder de Witt
2. Tabish Rashid
3. Gregory Farquhar
4. Shimon Whiteson

Citations in the corpus

• Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “MAGNet (Malysheva et al., 2018) allow agents to centrally learn a shared graph structure that encodes the relevance of individual environment entities to each agent.”
• A survey of multi-agent deep reinforcement learning with communication (2022, unknown): “CommNet [48]; BiCNet [50]; MS-MARL-GCM [88]; ATOC [39]; DGN [81]; TarMAC [89]; MAGNet-SA-GS-MG [40]; MADDPG-M [41]; IC3Net [49]; MD-MADDPG [42]; DCC-MD [82]; Agent-Entity Graph [74]; GA-Comm [83]; LSC [77]; Neur-Comm [84]; IP [85]; I2C [79]; IS [90]; HAMMER [91]; MAGIC [92]; FlowComm [93]; GAXNet [8”
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning (2020, unknown): “, 2018), represent relations (Zambaldi et al., 2018; Malysheva et al., 2018; Yang et al., 2019) and extract communication channels (Jiang & Lu, 2018).”
• Graph Convolutional Reinforcement Learning (2018, background): “MAGnet (Malysheva et al., 2018) learns relevance information in the form of a relevance graph, where relation weights are learned by pre-defined loss function based on heuristic rules, but relation weights in DGN are learned by directly minimizing the temporal-difference error of value function…”