TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

By Peng Sun, Xinghai Sun, Lei Han, Jiechao Xiong, Qing Wang, Bo Li, Yang Zheng, Ji Liu, Yongsheng Liu, Han Liu and 1 more. 2018

78 citations recorded by the source corpus.

Starcraft II (SC2) is widely considered as the most challenging Real Time Strategy (RTS) game. The underlying challenges include a large observation space, a huge (continuous and infinite) action space, partial observations, simultaneous move for all players, and long horizon delayed rewards for local decisions. To push the frontier of AI research, Deepmind and Blizzard jointly developed the StarCraft II Learning Environment (SC2LE) as a testbench of complex decision making systems. SC2LE provides a few mini games such as MoveToBeacon, CollectMineralShards, and DefeatRoaches, where some AI agents have achieved the performance level of human professional players. However, for full games, the current AI agents are still far from achieving human professional level performance. To bridge this gap, we present two full game AI agents in this paper - the AI agent TStarBot1 is based on deep reinforcement learning over a flat action structure, and the AI agent TStarBot2 is based on hard-coded rules over a hierarchical action structure. Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting. To the best of our knowledge, this is the first public work to investigate AI agents that can defeat the built-in AI in the StarCraft II full game.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The StarCraft Multi-Agent Challenge (2019, neutral): “Other work focuses on playing the full game of StarCraft, including macromanagement and tactics (Pang et al., 2018; Sun et al., 2018; DeepMind, 2019).”
A Survey of Deep Reinforcement Learning in Video Games (2019, methodology): “[99] develop the DRL based agent TStarBot, which uses flat action structure.”
On Reinforcement Learning for Full-length Game of StarCraft (2018, background): “ation-based reinforcement learning algorithm, which achieved good results on these mini-games. But the results on the full-length game are still not reported in this paper. Recently, there is a paper (15) about the full-length game of StarCraft II, which achieves good results. Our performance is clo”
Hierarchical Decision Making by Generating and Following Natural Language Instructions (2019, unknown): “For full games, [23] shows that it is possible to train an end-to-end agent on a small-scaled RTS game with predeﬁned macro actions, and TStarBot [20] applies this idea to StarCraft II and shows that the resulting agent can beat carefully-designed, and even cheating rule-based AI.”
Grandmaster level in StarCraft II using multi-agent reinforcement learning (2019, methodology): “Some of the most recent approaches use reinforcement learning to play the full game, with hand-crafted, high-level actions 48 , or rule-based systems with machine learning incrementally replacing components 43 .”
LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning (2019, methodology): “StarCraft II is a popular real-time strategy game and it has been studied under MARL settings [9, 10, 7, 36, 37].”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “transportation schedule [22, 53, 159, 253], traffic signal control [50, 59], energy management [180], wireless security [172], satellite docking [76], edge computing [299], chemical processes [241], video games [28, 153, 192, 211, 262, 300], board games of Go, shogi, chess and arcade game PAC-MAN [1”