MazeBase: A Sandbox for Learning from Games

By Sainbayar Sukhbaatar, Arthur Szlam, Gabriel Synnaeve, Soumith Chintala, Rob Fergus. 2015

80 citations recorded by the source corpus.

This paper introduces MazeBase: an environment for simple 2D games, designed as a sandbox for machine learning approaches to reasoning and planning. Within it, we create 10 simple games embodying a range of algorithmic tasks (e.g. if-then statements or set negation). A variety of neural models (fully connected, convolutional network, memory network) are deployed via reinforcement learning on these games, with and without a procedurally generated curriculum. Despite the tasks' simplicity, the performance of the models is far from optimal, suggesting directions for future development. We also demonstrate the versatility of MazeBase by using it to emulate small combat scenarios from StarCraft. Models trained on the MazeBase version can be directly applied to StarCraft, where they consistently beat the in-game AI.

Subfields: Artificial Intelligence, Sociology and Political Science, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Reinforcement Learning

Citations in the corpus

ShapeWorld - A new test methodology for multimodal language understanding (2017, methodology): “The MazeBase game environment (Sukhbaatar et al., 2015) uses language as a mean to represent the game world.”
Dialog-based Language Learning (2016, background): “The input is the last utterance of the dialog, x, as well as a set of memories (context) (c1, . . . , cN ) which can encode both short-term memory, e.g. recent previous utterances and replies, and long-term memories, e.g. facts that could be useful for answering questions.”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, methodology): “The following experiments explore our self-play approach on a variety of tasks, both continuous and discrete, from the Mazebase [21], RLLab [5], and StarCraft [23] environments.”
Control of Memory, Active Perception, and Action in Minecraft (2016, methodology): “In addition, there are deep RL approaches to tasks other than Atari such as learning algorithms (Zaremba et al., 2016) and text-based games (Sukhbaatar et al., 2015a; Narasimhan et al., 2015).”
Deep Successor Reinforcement Learning (2016, methodology): “In this section, we demonstrate the properties of our approach on MazeBase [37], a grid-world environment, and the Doom game engine [13].”
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent (2017, neutral): “Grounding language in an interactive environment is an active area of research, however a number of recent works employ synthetic, templated language only (Sukhbaatar et al., 2015; Yu et al., 2017; Bordes et al., 2010; Hermann et al., 2017; Mikolov et al., 2015; Chaplot et al., 2017).”
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning (2017, methodology): “We implemented actor-critic method with 16 CPU threads based on Sukhbaatar et al. (2015).”
Disentangling the independently controllable factors of variation by interacting with the world (2018, methodology): “We use MazeBase [Sukhbaatar et al., 2015] to assess the performance of our approach.”
TextWorld: A Learning Environment for Text-based Games (2018, methodology): “The MazeBase environment [Sukhbaatar et al., 2015], possibly the sandbox framework most similar to TextWorld, enables researchers to generate two-dimensional grid-based games.”
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning (2018, background): “General-purpose RL testbeds such as the Arcade Learning Environment (Bellemare et al., 2013), DM-30 (Espeholt et al., 2018), and MazeBase (Sukhbaatar et al., 2015) do not assume a human-in-the-loop setting.”
Policy Networks with Two-Stage Training for Dialogue Systems (2016, background): “Two recent examples are AlphaGo by DeepMind (Silver et al., 2016) and MazeBase by Facebook AI (Sukhbaatar et al., 2016).”
Independently Controllable Factors (2017, methodology): “3 Experiments on MazeBase We use MazeBase (Sukhbaatar et al., 2015) to assess the performance of our continuous embeddings approach on a more complex and well-known environment.”
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications (2021, background): “MazeBaze [44] is a highly configurable, open source, 2D learning environment that is suitable for cooperative and competitive tasks.”
ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games (2017, background): “, μRTS [23], MazeBase [29]) are fast and simple, often coming with various baselines, but often with much simpler dynamics than RTS games.”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “RL approaches [68,1,20,33,44,92,62,86] have been proposed to address navigation in synthetic layouts like mazes, arcade games and other visual environments [100,8,47,54,43,84].”
Interactive Grounded Language Acquisition and Generalization in a 2D World (2018, methodology): “Our XWORLD is similar to the 2D block world MaseBase (Sukhbaatar et al., 2016).”
Virtual Embodiment: A Scalable Long-Term Strategy for Artificial Intelligence Research (2016, background): “Our position is very much aligned with recent proposals for new directions in AI research [30, 31, 32, 33].”