StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning

By Kun Shao, Yuanheng Zhu, Dongbin Zhao. 2018

188 citations recorded by the source corpus.

Real-time strategy games have been an important field of game artificial intelligence in recent years. This paper presents a reinforcement learning and curriculum transfer learning method to control multiple units in StarCraft micromanagement. We define an efficient state representation, which breaks down the complexity caused by the large state space in the game environment. Then, a parameter sharing multi-agent gradient-descent Sarsa($\lambda$) algorithm is proposed to train the units. The learning policy is shared among our units to encourage cooperative behaviors. We use a neural network as a function approximator to estimate the action–value function, and propose a reward function to help units balance their move and attack. In addition, a transfer learning method is used to extend our model to more difficult scenarios, which accelerates the training process and improves the learning performance. In small-scale scenarios, our units successfully learn to combat and defeat the built-in AI with 100% win rates. In large-scale scenarios, the curriculum transfer learning method is used to progressively train a group of units, and it shows superior performance over some baseline methods in target scenarios. With reinforcement learning and curriculum transfer learning, our units are able to learn appropriate strategies in StarCraft micromanagement scenarios.

Subfields: Artificial Intelligence, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

Citations in the corpus

The StarCraft Multi-Agent Challenge (2019, background): “…has thus been steadily appearing (Usunier et al., 2016; Foerster et al., 2017, 2018a; Rashid et al., 2018; Nardelli et al., 2018; Hu et al., 2018; Shao et al., 2018; Foerster et al., 2018b) with the release of TorchCraft (Synnaeve et al., 2016) and SC2LE (Vinyals et al., 2017), interfaces to…”
Grandmaster level in StarCraft II using multi-agent reinforcement learning (2019, unknown): “Reinforcement learning has also been studied to control units in the game 7,34,42–44 , and imitation learning has been proposed to learn unit and building compositions 45 .”
Boosted Curriculum Reinforcement Learning (2022, background): “Curriculum RL is a biologically inspired approach that frames the problem of learning a complex target task into learning a sequence of simpliﬁed, increasingly difﬁcult, versions of it (Florensa et al., 2017; Shao et al., 2018; Ivanovic et al., 2019; Narvekar et al., 2020).”
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, methodology): “hod Curriculum Sequencer Curriculum Adaptivity Evaluation Metric Publication Status Clegg et al. (2017) domain experts sequence policies domain experts static asymptotic, time to threshold conference Shao et al. (2018) domain experts sequence task model domain experts static asymptotic, total reward”
A Survey of Deep Reinforcement Learning in Video Games (2019, methodology): “[94] follow decentralized micromanagement task, and propose parameter sharing multi-agent gradient descent SARSA(λ) (PSMAGDS) method.”