RODE: Learning Roles to Decompose Multi-Agent Tasks

By Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson, Chongjie Zhang. 2020

286 citations recorded by the source corpus.

Role-based learning holds the promise of achieving scalable multi-agent learning by decomposing complex tasks using roles. However, it is largely unclear how to efficiently discover such a set of roles. To solve this problem, we propose to first decompose joint action spaces into restricted role action spaces by clustering actions according to their effects on the environment and other agents. Learning a role selector based on action effects makes role discovery much easier because it forms a bi-level learning hierarchy -- the role selector searches in a smaller role space and at a lower temporal resolution, while role policies learn in significantly reduced primitive action-observation spaces. We further integrate information about action effects into the role policies to boost learning efficiency and policy generalization. By virtue of these advances, our method (1) outperforms the current state-of-the-art MARL algorithms on 10 of the 14 scenarios that comprise the challenging StarCraft II micromanagement benchmark and (2) achieves rapid transfer to new environments with three times the number of agents. Demonstrative videos are available at this https URL .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (2021, methodology): “…to recent academic progress and literature in MARL which has largely focused developing off-policy learning frameworks such as MADDPG [22] and value-decomposed Q-learning [32, 27]; methods in these frameworks have yielded state-of-the-art results on a wide range of multi-agent benchmarks [36, 37].”
Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (2020, methodology): “Wang et al. (2021) introduce a method to decompose action spaces into roles, which they show can transfer to tasks with larger numbers of agents by grouping new actions into existing clusters.”
The Surprising Effectiveness of MAPPO in Cooperative, Multi-Agent Games (2021, methodology): “13 shows the evaluation win rate of MAPPO compared to RODE and QMix, in the maps in which RODE’s results are available.”