Learning Attentional Communication for Multi-Agent Cooperation
By Jiechuan Jiang, Zongqing Lu. 2018
609 citations recorded by the source corpus.
Communication could potentially be an effective way for multi-agent cooperation. However, information sharing among all agents or in predefined communication architectures that existing methods adopt can be problematic. When there is a large number of agents, agents cannot differentiate valuable information that helps cooperative decision making from globally shared information. Therefore, communication barely helps, and could even impair the learning of multi-agent cooperation. Predefined communication architectures, on the other hand, restrict communication among agents and thus restrain potential cooperation. To tackle these difficulties, in this paper, we propose an attentional communication model that learns when communication is needed and how to integrate shared information for cooperative decision making. Our model leads to efficient and effective communication for large-scale multi-agent cooperation. Empirically, we show the strength of our model in a variety of cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies than existing methods.
Subfields: Artificial Intelligence, Computer Networks and Communications, Sociology and Political Science, LLM Agents & Tool Use, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Sergey Levine
- Dacheng Tao
- Yoshua Bengio
- Trevor Darrell
- Ruslan Salakhutdinov
- Li Fei-Fei
- Jianfeng Gao
In Computer Networks and Communications
- Afshin Oroojlooyjadid
- Davood Hajinezhad
- Tonghan Wang
- Chongjie Zhang
- Xiujun Ma
- Yuhang Zhao
- Bangwei Li
- Yongyuan Liang
In Sociology and Political Science
- Davood Hajinezhad
- Afshin Oroojlooyjadid
- Xiujun Ma
- Yuhang Zhao
- Bangwei Li
- Yongyuan Liang
- Jinkyoo Park
- Hayong Shin
In LLM Agents & Tool Use
- Tonghan Wang
- Chongjie Zhang
- Davood Hajinezhad
- Afshin Oroojlooyjadid
- Bangwei Li
- Yongyuan Liang
- Xiujun Ma
- Yuhang Zhao
In Reinforcement Learning
- Chongjie Zhang
- Shimon Whiteson
- Tonghan Wang
- Zongqing Lu
- Dhruv Batra
- Silvio Savarese
- Heng Dong
- Jiechuan Jiang
Citations in the corpus
- Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication (2020, neutral): “The attentional communication model (ATOC) allows agents to select and communicate to “collaborators” [11].”
- Learning Multi-Arm Manipulation Through Collaborative Teleoperation (2020, background): “Multi-Agent Reinforcement Learning: Multi-Agent Re-inforcement Learning [5, 35] in cooperative settings has been widely studied [11, 12, 14, 21, 26, 34], and applied to domains such as video games [28] and visual question answering [9].”
- QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, methodology): “Other related work includes CommNet (Sukhbaatar et al., 2016), DIAL (Foerster et al., 2016), ATOC (Jiang & Lu, 2018), and SCHEDNET (Kim et al., 2019), which exploit inter-agent communication in execution.”
- MAVEN: Multi-Agent Variational Exploration (2019, neutral): “Yet another interesting direction would be to condition the latent distribution on the joint state space at each time step and transmit it across the agents to get a low communication cost, centralised execution policy and compare its merits to existing methods [45, 11, 23].”
- From Few to More: Large-scale Dynamic Multiagent Curriculum Learning (2019, background): “Furthermore, in large-scale multiagent systems, the dynamics and stochasticity of the environment become more complex, which makes it more challenging to achieve coordination among agents (Singh, Jain, and Sukhbaatar 2019; Yang et al. 2018a; Jiang and Lu 2018).”
- Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control (2019, background): “[7] propose an attentional communication model that learns when the communication is required and how to aggregate the shared information.”
- Actor-Attention-Critic for Multi-Agent Reinforcement Learning (2018, background): “Note that communication is highly restricted and different from centralized policy approaches Jiang & Lu (2018), which allow for free transfer of continuous information among policies.”
- XOC: Explainable Observer-Classifier for Explainable Binary Decisions (2019, background): “Learning to communicate in a multi-agent setting has gained interest with the emergence of deep reinforcement learning [9, 10, 22, 4, 15, 7, 6].”
- ROMA: Multi-Agent Reinforcement Learning with Emergent Roles (2020, background): “…et al., 2018; Yang et al., 2018; Grover et al., 2018; Kipf et al., 2018) and explicit communication (Sukhbaatar et al., 2016; Hoshen, 2017; Jiang & Lu, 2018; Singh et al., 2019; Das et al., 2019; Singh et al., 2019; Kim et al., 2019) are studied to model the dependence between the…”
- Learning to Schedule Communication in Multi-agent Reinforcement Learning (2019, background): “Recently, Jiang & Lu (2018) proposes an attentional communication model that allows some agents who request additional information from others to gather observation from neighboring agents.”
- LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning (2019, background): “Examples include a wide range of approaches on designing effective centralized MARL architectures [5, 6, 7, 8].”
- When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning (2019, background): “Recent work from Jiang and Lu (2018) features the attentional communication model (ATOC), allowing multi-agent systems to learn to communicate efficiently when they are in a partially observable environment.”
- Learning to Coordinate Manipulation Skills via Skill Behavior Diversification (2020, background): “For cooperative tasks, communication mechanisms (Sukhbaatar et al., 2016; Peng et al., 2017; Jiang & Lu, 2018), sharing policy parameters (Gupta et al., 2017), and decentralized actor with centralized critic (Lowe et al., 2017; Foerster et al., 2018) have been actively used.”
- Learning Distilled Collaboration Graph for Multi-Agent Perception (2021, background): “For example, CommNet [32] adopted the averaging operation to achieve continuous communication in multi-agent system; VAIN [11] considered an attention mechanism to determine which agents would share information; ATOC [12] exploited an attention unit to determine what information to share with other ”
- Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems (2021, methodology): “We re-train VACL on the same environment configuration (i.e., same agent/landmark size and room size) as ATOC for a fair comparison with the number reported in [13].”
- Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning (2020, neutral): “There are also other works studying large-scale MARL recently (Lin et al., 2018; Jiang & Lu, 2018; Yang et al., 2018; Suarez et al., 2019), which typically simplify the problem by weight sharing and taking only local observations.”
- A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “The main idea in this work is to use an attention mechanism [24, 54] to select relevant information for each agent during the training.”
- Graph Convolutional Reinforcement Learning (2018, background): “…costly and less helpful to take all other agents into consideration, because receiving a large amount of information requires high bandwidth and incurs high computational complexity, and agents cannot differentiate valuable information from globally shared information (Tan, 1993; Jiang & Lu, 2018).”
- TarMAC: Targeted Multi-Agent Communication (2018, background): “…agents to communicate with discrete symbols with the explicit goal to study emergence of language, our work operates in the paradigm of learning a continuous communication protocol in order to solve a downstream task (Sukhbaatar et al., 2016; Hoshen, 2017; Jiang & Lu, 2018; Singh et al., 2019).”