Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

By Amanpreet Singh, Tushar Jain, Sainbayar Sukhbaatar. 2018

319 citations recorded by the source corpus.

Learning when to communicate and doing that effectively is essential in multi-agent tasks. Recent works show that continuous communication allows efficient training with back-propagation in multi-agent scenarios, but have been restricted to fully-cooperative tasks. In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings. IC3Net controls continuous communication with a gating mechanism and uses individualized rewards foreach agent to gain better performance and scalability while fixing credit assignment issues. Using variety of tasks including StarCraft BroodWars explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases. Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

ROMA: Multi-Agent Reinforcement Learning with Emergent Roles (2020, background): “…et al., 2018; Grover et al., 2018; Kipf et al., 2018) and explicit communication (Sukhbaatar et al., 2016; Hoshen, 2017; Jiang & Lu, 2018; Singh et al., 2019; Das et al., 2019; Singh et al., 2019; Kim et al., 2019) are studied to model the dependence between the decision-making processes of agents.”
Why Do Multi-Agent LLM Systems Fail? (2025, unknown): “…the usage of MAST as a debugging tool, where we measure the failure modes in the system before applying any of the interventions, and then after applying the interventions we discuss below, and show that MAST can guide the intervention process as well as capture the improvements of augmentations.”
Multi-agent deep reinforcement learning: a survey (2021, background): “In contrast to former works, which are limited to the fully-cooperative setting, Singh et al. (2019) considered mixed settings where each agent owns an individual (IC3Net), where agents learn when to exchange information using a gating mechanism that blocks incoming communication requests if…”
TarMAC: Targeted Multi-Agent Communication (2018, methodology): “Jiang & Lu (2018); Singh et al. (2019) also operate in a decentralized execution setting and use an attentional communication mechanism, but in contrast to our work, they use attention to decide when to communicate, not who to communicate with.”
Multi-Agent Actor-Critic with Hierarchical Graph Attention Network (2019, unknown): “, interchanges messages) to achieve consensus among agents during executing phase [25, 17, 11, 23, 5].”
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning (2018, background): “, 2018), our IAC agents have access to individual rewards that resolve the credit assignment issue and improve performance (Singh et al., 2019).”
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning (2019, background): “Singh et al. (2019) proposed the Individualized Controlled Continuous Communication Model (IC3NET), which is an extension of CommNet in competitive setting.”
A survey of multi-agent deep reinforcement learning with communication (2022, unknown): “Among them, MAGIC [92] achieved higher performance on Traffic Junction with local rewards compared to two early works, CommNet [48], IC3Net [49], and one recent work, GA-Comm [83].”
When2com: Multi-Agent Perception via Communication Graph Grouping (2020, background): “Recent works have proposed to construct communication groups based on pre-deﬁned rules [18, 17] or a uniﬁed communication network [33, 34, 15, 28, 33, 6].”
Learning Distilled Collaboration Graph for Multi-Agent Perception (2021, background): “As core components in a multi-agent system, communication strategy among agents has been actively studied in previous works [30, 7, 31].”
Who2com: Collaborative Perception via Learnable Handshake Communication (2020, background): “Early attempts utilized a pre-defined communication protocol to propagate the information across agents [37, 39, 27], while dynamics of environments and a varied number of agents gave rise to the development of learnable mulit-agent communication [36, 11, 2, 14, 10, 29, 17, 35, 18, 7, 30, 9, 15, 19]”
A review of cooperative multi-agent deep reinforcement learning (2019, background): “Among those, StarCraft II [108] has achieved a lot of attention in the recent years [35, 140, 121, 37, 106, 101].”
Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps (2022, background): “The communication strategy in multi-agent systems has been widely studied [20].”