Action Semantics Network: Considering the Effects of Actions in Multiagent Systems

By Weixun Wang, Tianpei Yang, Yong Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, Yang Gao. 2019

46 citations recorded by the source corpus.

In multiagent systems (MASs), each agent makes individual decisions but all of them contribute globally to the system evolution. Learning in MASs is difficult since each agent's selection of actions must take place in the presence of other co-learning agents. Moreover, the environmental stochasticity and uncertainties increase exponentially with the increase in the number of agents. Previous works borrow various multiagent coordination mechanisms into deep learning architecture to facilitate multiagent coordination. However, none of them explicitly consider action semantics between agents that different actions have different influences on other agents. In this paper, we propose a novel network architecture, named Action Semantics Network (ASN), that explicitly represents such action semantics between agents. ASN characterizes different actions' influence on other agents using neural networks based on the action semantics between them. ASN can be easily combined with existing deep reinforcement learning (DRL) algorithms to boost their performance. Experimental results on StarCraft II micromanagement and Neural MMO show ASN significantly improves the performance of state-of-the-art DRL approaches compared with several network architectures.

Subfields: Artificial Intelligence, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, methodology): “…introduction, SMAC and PyMARL have been used by a number of papers to evaluate and compare multi-agent RL methods (de Witt et al., 2018; Wang et al., 2019c; Du et al., 2019; Wang et al., 2019a; Zhang et al., 2019; Mahajan et al., 2019; Yao et al., 2019; Bhmer et al., 2019; Wang et al., 2019b).”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, methodology): “However, learning roles from scratch might not be easier than learning without roles, as directly ﬁnding an optimal decomposition suffers from the same problem as other CTDE learning methods – searching in the large joint space with substantial exploration (Wang et al., 2020c).”