TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

By Gabriel Synnaeve, Nantas Nardelli, Alex Auvolat, Soumith Chintala, Timothée Lacroix, Zeming Lin, Florian Richoux, Nicolas Usunier. 2016

112 citations recorded by the source corpus.

We present TorchCraft, a library that enables deep learning research on Real-Time Strategy (RTS) games such as StarCraft: Brood War, by making it easier to control these games from a machine learning framework, here Torch [9]. This white paper argues for using RTS games as a benchmark for AI research, and describes the design and components of TorchCraft.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Counterfactual Multi-Agent Policy Gradients (2017, methodology): “Our implementation uses TorchCraft (Synnaeve et al., 2016) and Torch 7 (Collobert et al., 2011).”
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, methodology): “A number of papers have established unit micromanagement in StarCraft as a benchmark for deep multi-agent RL. Usunier et al. (2017) present an algorithm using a centralised greedy MDP and ﬁrst-order optimisation which they evaluate on Starcraft: BroodWar (Synnaeve et al., 2016).”
Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games (2017, background): “Such game is considered as one of the most difficult games for computers with more possible states than Go game (Synnaeve et al. 2016).”
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (2017, methodology): “(2011), and we run our StarCraft experiments with TorchCraft (Synnaeve et al., 2016), a library that provides the functionality to enact the standard reinforcement learning step in StarCraft: BroodWar in a centralised way (which we extend to enable multi-agent control).”
The StarCraft Multi-Agent Challenge (2019, methodology): “The full games of StarCraft: BroodWar and StarCraft II have already been used as RL environments, due to the many interesting challenges inherent to the games (Synnaeve et al., 2016; Vinyals et al., 2017).”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, methodology): “Previous work (Usunier et al., 2017; Foerster et al., 2018; Peng et al., 2017) applied RL to the original version of Star-Craft: BW, which made use of the standard API or related wrappers (Synnaeve et al., 2016).”
Multi-Agent Common Knowledge Reinforcement Learning (2018, neutral): “This setting, illustrated in Figure 1 and formalised in section 5, arises in tasks such as robo-soccer (Genter et al., 2017) and multi-agent StarCraft Micromanagement (Synnaeve et al., 2016).”
StarCraft II: A New Challenge for Reinforcement Learning (2017, background): “The standard API for StarCraft thus far has been BWAPI [1], and related wrappers [32].”
AI2-THOR: An Interactive 3D Environment for Visual AI (2017, background): “A variety of benchmarks are based on games such as ATARI Learning Environment [4], ViZDoom [13], TorchCraft [19], ELF [20] or DeepMind Lab [3].”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, methodology): “The following experiments explore our self-play approach on a variety of tasks, both continuous and discrete, from the Mazebase [21], RLLab [5], and StarCraft [23] environments.”
Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, background): “This includes 3D understanding (DeepMind Lab (Beattie et al., 2016) and Malmo (Johnson et al., 2016)), real-time strategy decision (TorchCraft (Synnaeve et al., 2016) and ELF (Tian et al., 2017)), fast reaction (Atari (Bellemare et al., 2013)), long-term planning (Go, Chess), language and…”
Knowledge-Guided Agent-Tactic-Aware Learning for StarCraft Micromanagement (2018, background): “In recent years, StarCraft micromanagement [Ontan´on et al. , 2013; Wender and Watson, 2012; Synnaeve et al. , 2016] based on deep reinforcement learning (DRL) [Mnih et al. , 2015; Li, 2017; Silver et al. , 2016; Lillicrap et al. , 2015; Levine et al. , 2016; Mirowski et al. , 2017; Pasunuru and…”
Composable Planning with Attributes (2018, methodology): “Finally, we test our approach for planning a build order in StarCraft: Brood War (Synnaeve et al., 2016).”
Why Build an Assistant in Minecraft? (2019, background): “[80] Gabriel Synnaeve, Nantas Nardelli, Alex Auvolat, Soumith Chintala, Timothée Lacroix, Zeming Lin, Florian Richoux, and Nicolas Usunier.”
Value Propagation Networks (2018, methodology): “We use TorchCraft (Synnaeve et al., 2016) to setup the environment and extract the position and type of units randomly spawned in the scenario.”
Multi-Target Embodied Question Answering (2019, methodology): “Examples include ViZDoom [20], TorchCraft [30] and DeepMind Lab [5].”
Hierarchical Decision Making by Generating and Following Natural Language Instructions (2019, unknown): “However, manually designed macro-actions typically do not match strong human performance, because of the unbounded space of possible strategies [21, 25].”
An Introduction to Deep Reinforcement Learning (2018, unknown): “, 2017) and “Starcraft: Broodwar" (Wender and Watson, 2012; Synnaeve et al., 2016) provide similar benefits in exploring lifelong-learning, curriculum learning, and other related hierarchical approaches.”
Synthetic Data for Deep Learning (2019, methodology): “All of the above does not look too much in line with the general topic of synthetic data: while game environments are certainly “synthetic”, there is usually no goal to transfer, say, an RL agent playing StarCraft to a real armed conflict (thankfully).”