Graph Convolutional Reinforcement Learning

By Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu. 2018

435 citations recorded by the source corpus.

Learning to cooperate is crucially important in multi-agent environments. The key is to understand the mutual interplay between agents. However, multi-agent environments are highly dynamic, where agents keep moving and their neighbors change quickly. This makes it hard to learn abstract representations of mutual interplay between agents. To tackle these difficulties, we propose graph convolutional reinforcement learning, where graph convolution adapts to the dynamics of the underlying graph of the multi-agent environment, and relation kernels capture the interplay between agents by their relation representations. Latent features produced by convolutional layers from gradually increased receptive fields are exploited to learn cooperation, and cooperation is further improved by temporal relation regularization for consistency. Empirically, we show that our method substantially outperforms existing methods in a variety of cooperative scenarios.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Coordination Graphs (2019, methodology): “Current MARL literature uses GNN therefore either as independent (IQL) value functions (Jiang et al., 2020; Luo et al., 2019) or as a joint critic in actor-critic frameworks (Tacchetti et al.”
A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “[83] assume a graph connection among the”
The Emergence of Individuality in Multi-Agent Reinforcement Learning (2020, background): “Communication methods [19, 20, 21] share information between agents for better cooperation.”
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, methodology): “For training, we use the following hyperparameters for GPL and all proposed baselines: • K = 5 for the low rank factorization of pairwise utility terms in GPL. • 8 attention heads were used for DGN, GNN-QL and GNN-QL-AM.”
FOP: Factorizing Optimal Joint Policy of Maximum-Entropy Multi-Agent Reinforcement Learning (2021, background): “…et al., 2019), optimizing both efﬁciency and fairness (Jiang & Lu, 2019), incorporating agent communication (Sukhbaatar et al., 2016; Das et al., 2019; Ding et al., 2020), considering relation between agents (Jiang et al., 2020) or their underlying network (Zhang et al., 2018; Qu et al., 2019).”
ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind (2021, background): “In previous works (Jiang et al., 2019), there is only one global graph that collects all the observations as node features.”
A survey of multi-agent deep reinforcement learning with communication (2022, neutral): “CommNet [48]; BiCNet [50]; MS-MARL-GCM [88]; ATOC [39]; DGN [81]; TarMAC [89]; MAGNet-SA-GS-MG [40]; MADDPG-M [41]; IC3Net [49]; MD-MADDPG [42]; DCC-MD [82]; Agent-Entity Graph [74]; GA-Comm [83]; LSC [77]; Neur-Comm [84]; IP [85]; I2C [79]; IS [90]; HAMMER [91]; MAGIC [92]; FlowComm [93]; GAXNet…”