From Few to More: Large-scale Dynamic Multiagent Curriculum Learning
By Weixun Wang, Tianpei Yang, Y. Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, Yang Gao. 2019
141 citations recorded by the source corpus.
A lot of efforts have been devoted to investigating how agents can learn effectively and achieve coordination in multiagent systems. However, it is still challenging in large-scale multiagent settings due to the complex dynamics between the environment and agents and the explosion of state-action space. In this paper, we design a novel Dynamic Multiagent Curriculum Learning (DyMA-CL) to solve large-scale problems by starting from learning on a multiagent scenario with a small size and progressively increasing the number of agents. We propose three transfer mechanisms across curricula to accelerate the learning process. Moreover, due to the fact that the state dimension varies across curricula, and existing network structures cannot be applied in such a transfer setting since their network input sizes are fixed. Therefore, we design a novel network structure called Dynamic Agent-number Network (DyAN) to handle the dynamic size of the network input. Experimental results show that DyMA-CL using DyAN greatly improves the performance of large-scale multiagent learning compared with state-of-the-art deep reinforcement learning approaches. We also investigate the influence of three transfer mechanisms across curricula through extensive simulations.
Subfields: Artificial Intelligence, Reinforcement Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shimon Whiteson
- Bei Peng
- Christian Schröder de Witt
- Wendelin Böhmer
- Tabish Rashid
- Jakob N. Foerster
- Gregory Farquhar
- Matthew E. Taylor
In Reinforcement Learning
- Shimon Whiteson
- Bei Peng
- Wendelin Böhmer
- Christian Schröder de Witt
- Tabish Rashid
- Jakob N. Foerster
- Gregory Farquhar
- Chongjie Zhang
In Meta & Continual Learning
- Peter Stone
- Matthew E. Taylor
- Sanmit Narvekar
- Jivko Sinapov
- Shimon Whiteson
- Matteo Leonetti
- Fei Sha
- Shariq Iqbal
Citations in the corpus
- Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, methodology): “…introduction, SMAC and PyMARL have been used by a number of papers to evaluate and compare multi-agent RL methods (de Witt et al., 2018; Wang et al., 2019c; Du et al., 2019; Wang et al., 2019a; Zhang et al., 2019; Mahajan et al., 2019; Yao et al., 2019; Bhmer et al., 2019; Wang et al., 2019b).”
- Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems (2021, background): “[18, 31] propose population curriculum, which trains entity-invariant policies over training tasks with a growing number of agents.”
- Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, background): “Curriculum learning approaches have also been proposed for cooperative multi-agent systems (Wang et al., 2020; Yang et al., 2020).”
- Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (2020, background): “rk based models for handling variable sized inputs and focus on learning curricula to progress on increasingly large/challenging settings (Agarwal et al., 2019; Long et al., 2020; Baker et al., 2019; Wang et al., 2020) In contrast to these curriculum learning approaches, we focus on training simulta”