Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

By Arthur Juliani, Ahmed Khalifa, Vincent-Pierre Berges, Jonathan Harper, Jonathan Harper, Hunter Henry, Adam Crespi, Julian Togelius, Danny B. Lange. 2019

151 citations recorded by the source corpus.

The rapid pace of recent research in AI has been driven in part by the presence of fast and challenging simulation environments. These environments often take the form of games; with tasks ranging from simple board games, to competitive video games. We propose a new benchmark - Obstacle Tower: a high fidelity, 3D, 3rd person, procedurally generated environment. An agent in Obstacle Tower must learn to solve both low-level control and high-level planning problems in tandem while learning from pixels and a sparse reward signal. Unlike other benchmarks such as the Arcade Learning Environment, evaluation of agent performance in Obstacle Tower is based on an agent's ability to perform well on unseen instances of the environment. In this paper we outline the environment and provide a set of baseline results produced by current state-of-the-art Deep RL methods as well as human players. These algorithms fail to produce agents capable of performing near human level.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “We, among many others [93, 78, 94, 95, 96], have explored a variety of o.”
Leveraging Procedural Generation to Benchmark Reinforcement Learning (2019, methodology): “While previously mentioned studies (Zhang et al., 2018c; Cobbe et al., 2019; Justesen et al., 2018; Juliani et al., 2019) reveal intriguing trends, it is hard to draw general conclusions from so few environments.”
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems (2019, neutral): “Other recent work on the Obstacle Tower challenge domain (Juliani et al., 2019) is similar to our task suite in this regard.”
Unity: A General Platform for Intelligent Agents (2018, background): “We also discuss the Obstacle Tower benchmark (Juliani et al., 2019) which serves as an example of the degree of environmental complexity achievable on Unity.”
ORRB - OpenAI Remote Rendering Backend (2019, background): “ORRB renderer is a standalone binary built with the Unity game engine.”
The Measure of Intelligence (2019, unknown): “This proposed setup is thus markedly different from e.g. CoinRun [17] or Obstacle Tower [49], where the evaluation environments are not alternative games, but only levels of the same game (local generalization, or generalization to known unknowns), randomly sampled from a level generator which is kn”
Neuroevolution of self-interpretable agents (2020, background): “[3, 32, 45, 46, 52, 58, 70, 87, 102, 106]) that approach the generalization problem, and future work will continue to develop on these ideas to push the generalization abilities proposed in more general domains [7, 16, 17, 49, 59, 70, 105].”
L2Explorer: A Lifelong Reinforcement Learning Assessment Environment (2022, methodology): “2018) and the procedurally generated Obstacle Tower (Juliani et al. 2019) environment.”
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning (2021, background): “…; with the community highlighting that deep RL agents tend to overﬁt to the environments they are trained on (Cobbe et al., 2019; Witty et al., 2018; Farebrother et al., 2018; Juliani et al., 2019; Kostrikov et al., 2020; Song et al., 2019; Justesen et al., 2018; Packer et al., 2018).”
Generalization and Regularization in DQN (2018, background): “There are also some proposals for evaluating generalization in RL through procedurally generated or parametrized environments (e.g., Finn et al., 2017; Juliani et al., 2019; Justesen et al., 2018; Whiteson et al., 2011; Witty et al., 2018).”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, background): “RL is also frequently evaluated on video game-like environments, most commonly Atari [3], among others [9, 8, 70, 2, 25, 23, 17, 10, 34].”
Prioritized Level Replay (2020, background): “, 2018), the Obstacle Tower Challenge (Juliani et al., 2019), the Procgen Benchmark (Cobbe et al.”
Human-Timescale Adaptation in an Open-Ended Task Space (2023, background): “Indeed, in the past few years a series of challenging PCG environments have been proposed (Juliani et al., 2019; Cobbe et al., 2020; K¨uttler et al., 2020; Samvelyan et al., 2021; Chevalier-Boisvert et al., 2018; Hafner, 2022; Deitke et al., 2022), mostly focusing on testing and improving…”
The NetHack Learning Environment (2020, methodology): “More recent work has produced game-like environments with procedurally generated elements, such as the Procgen Benchmark [18], MazeExplorer [30], and the Obstacle Tower environment [38].”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “Prior works include maze-style worlds [121, 129, 61], purely text-based game [69], grid worlds [21, 16], browser/GUI-based environments [108, 124], and indoor simulators for robotics [1, 107, 114, 34, 110, 99, 89].”