ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

By Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, C. Lawrence Zitnick. 2017

131 citations recorded by the source corpus.

In this paper, we propose ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research. Using ELF, we implement a highly customizable real-time strategy (RTS) engine with three game environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a miniature version of StarCraft, captures key game dynamics and runs at 40K frame-per-second (FPS) per core on a Macbook Pro notebook. When coupled with modern reinforcement learning methods, the system can train a full-game bot against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition, our platform is flexible in terms of environment-agent communication topologies, choices of RL methods, changes in game parameters, and can host existing C/C++-based game environments like Arcade Learning Environment. Using ELF, we thoroughly explore training parameters and show that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than $70\%$ of the time in the full game of Mini-RTS. Strong performance is also achieved on the other two games. In game replays, we show our agents learn interesting strategies. ELF, along with its RL platform, is open-sourced at this https URL.

Subfields: Artificial Intelligence, Developmental and Educational Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

Citations in the corpus

StarCraft II: A New Challenge for Reinforcement Learning (2017, neutral): “Simplified versions of RTS games have also been developed for AI research, most notably microRTS3 or the more recent ELF [34].”
Dopamine: A Research Framework for Deep Reinforcement Learning (2018, background): “…conclusions extend to similar environments, including continuous control (Duan et al., 2016; Tassa et al., 2018), ﬁrst-person environments (Kempka et al., 2016; Beattie et al., 2016; Johnson et al., 2016), and to some extent real-time strategy games (Tian et al., 2017; Vinyals et al., 2017).”
Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, background): “, 2016) and ELF (Tian et al., 2017)), fast reaction (Atari (Bellemare et al.”
Ray: A Distributed Framework for Emerging AI Applications (2017, background): “As a result, researchers and practitioners today build one-off systems for specialized RL applications [58, 41, 54, 44, 49, 5].”
MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments (2017, neutral): “The ELF platform (Tian et al., 2017) (Johnson et al., 2016).”
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero (2019, methodology): “We now describe the system and software design of ELF OpenGo, which builds upon the DarkForest Go engine (Tian & Zhu, 2015) and the ELF reinforcement learning platform (Tian et al., 2017).”
Idiosyncrasies and challenges of data driven learning in electronic trading (2018, unknown): “An exciting development is the emergence of open source RL frameworks such as OpenAI base-lines [Dhariwal et al., 2017], ELF [Tian et al., 2017], Horizon [Gauci et al., 2018], dopamine [Belle-mare et al., 2018], TRFL [Deepmind, 2018] and Ray RLlib [Moritz et al., 2017].”
On Reinforcement Learning for Full-length Game of StarCraft (2018, background): “In order to improve the ability of reinforcement learning, complex strategic games like StarCraft has become the perfect simulation environment for many institutions such as DeepMind ( 18 ), FAIR ( 15 ), and Alibaba ( 8 ).”
TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game (2018, background): “2) is similar to the work of [30], where the authors adopt macro action for a customized mini RTS game.”
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP (2019, methodology): “3 in Vaswani et al. (2017)).”
Decoupling Dynamics and Reward for Transfer Learning (2018, background): “One challenge is the massive amounts of data required to learn an optimal behavior; this can be alleviated by using a highfidelity simulator or game engine (Brockman et al., 2016; Tian et al., 2017), but there are many real-world domains where this is not available (Kober et al.”
Hierarchical Decision Making by Generating and Following Natural Language Instructions (2019, unknown): “We develop an RTS game based on the MiniRTS framework, aspiring to make it intuitive for humans, while still providing a signiﬁcant challenge to machines due to extremely high-dimensional observation and actions spaces, partial observability, and non-stationary environment dynamics imposed by the op”
Adaptive Power System Emergency Control Using Deep Reinforcement Learning (2019, methodology): “Open platforms and/or tools, such as OpenAI Gym [24] and ELF [25] have been proven to be significantly beneficial for developing and comparing RL algorithms in games and robotics.”
RecSim: A Configurable Simulation Platform for Recommender Systems (2019, background): “ELF [Tian et al., 2017] is a platform that allows conﬁguration of real-time strategy games (RTSs) to support the development of new RL methods to overcome the challenges of doing research with commercial games (e.g., by allowing access to internal game state).”
An Introduction to Deep Reinforcement Learning (2018, unknown): “• ELF (Tian et al., 2017) is a research platform for deep RL, aimed mainly to real-time strategy games.”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, background): “…rest of the platforms are limited in the sense that they are built for speciﬁc tasks, such as ﬁrst person shooting [104], Real-Time Strategy (RTS) [93], vision understanding [76], in-door scene understanding [12, 15, 29, 80], and interaction [45, 81, 103], or speciﬁc games, such as driving [20,…”