OpenAI Gym
By Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba. 2016
5,596 citations recorded by the source corpus.
OpenAI Gym is a toolkit for reinforcement learning research. It includes a growing collection of benchmark problems that expose a common interface, and a website where people can share their results and compare the performance of algorithms. This whitepaper discusses the components of OpenAI Gym and the design decisions that went into the software.
Subfields: Developmental and Educational Psychology, Reinforcement Learning, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Developmental and Educational Psychology
- J. Urgen Schmidhuber
- Pieter Abbeel
- Sergey Levine
- David R Ha
- Shimon Whiteson
- Sergey I. Nikolenko
- Joelle Pineau
- Shangtong Zhang
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Ofir Nachum
- Shimon Whiteson
- Shixiang Shane Gu
- Yuxi Li
- Abhishek Gupta
In Evaluation & Benchmarks
- Sergey Levine
- Pieter Abbeel
- Abhinav Kumar Gupta
- Jitendra Malik
- Trevor Darrell
- Alexei A. Efros
- Li Fei-Fei
- Dhruv Batra
Citations in the corpus
- ShapeWorld - A new test methodology for multimodal language understanding (2017, methodology): “Various platforms following this paradigm have been released, mostly aimed at reinforcement learning: the Arcade Learning Environment / Atari 2600 games (Bellemare et al., 2013), OpenAI Gym (Brockman et al., 2016), DeepMind Lab (Beattie et al., 2016), Project Malmo (Johnson et al., 2016), to name a ”
- Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning (2019, methodology): “We evaluate our method on the OpenAI Gym benchmarks (Brockman et al., 2016), consisting of discrete and continuous control tasks.”
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, methodology): “Using Flask [41] and OpenAI Gym [5], we provide two parallel observation modes to render the state and instruction S × I → O: (1) HTML mode that contains the HTML of the web page, allowing for interaction in a web browser(Figure 1A), and (2) simple mode which strips away extraneous meta-data from ra”
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, methodology): “Atari Riverraid.”
- Measuring the Intrinsic Dimension of Objective Landscapes (2018, methodology): “We start with a simple classic control game CartPole − v0 in OpenAI Gym (Brockman et al., 2016).”
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (2017, methodology): “In addition to the C++ API, Python bindings are also provided, allowing the simulator to be easily used with deep learning frameworks such as Caffe [20] and TensorFlow [1], or within RL platforms such as ParlAI [33] and OpenAI Gym [8].”
- Deep Reinforcement Learning that Matters (2017, methodology): “These methods have shown promising results in continuous control MuJoCo domain tasks (Todorov, Erez, and Tassa 2012) from OpenAI Gym (Brockman et al. 2016).”
- Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Pseudocode for how our agent model is used in the OpenAI Gym [5] environment (right).”
- Surprising Negative Results for Generative Adversarial Tree Search (2018, methodology): “For this work, we also developed a new OpenAI gym (Brockman et al., 2016)-like interface for the latest Atari Learning Environment (ALE) (Machado et al., 2017), which supports different modes and difficulties for Atari games.”
- Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, methodology): “…many researchers have augmented the standard behavior of the ALE to ev luat the robustness of their agents and to discourage memorization (e.g., random frame skips, Brockman et al., 2016; injecting stochasticity, Hausknecht & Stone, 2015; no-ops, Mnih et al., 2015; human starts, Nair t l., 2015).”
- Generative Adversarial Imitation Learning (2016, background): “Each task comes with a true cost function, defined in the OpenAI Gym [5].”
- AutoML: A Survey of the State-of-the-Art (2019, background): “For example, OpenAI Gym [44] is a popular toolkit that provides various simulation environment, where the developers can concentrate on designing their algorithms, instead of struggling for generating data.”
- Weight Agnostic Neural Networks (2019, methodology): “The second task, BipedalWalker-v2 [11], is to guide a two-legged agent across randomly generated terrain.”
- Learning data augmentation policies using augmented random search (2018, background): “In particular, ARS has been proved to match or exceed state-of-the-art sample efficiency on Mu-JoCo locomotion benchmarks [6, 7].”
- RUDDER: Return Decomposition for Delayed Rewards (2018, methodology): “RUDDER is evaluated with respect to its learning time and achieved scores on Atari games of the Arcade Learning Environment (ALE) [9] and OpenAI Gym [13].”
- The Mirage of Action-Dependent Baselines in Reinforcement Learning (2018, background): “The standard continuous control benchmarks use a fixed time horizon (Duan et al., 2016; Brockman et al., 2016), yet most value function parameterizations are stationary, as though the task had infinite horizon.”
- CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, methodology): “Previous benchmarks proposed for RL mostly focused on the single task learning setting such as OpenAI Gym and DM control suite (Tassa et al., 2018; Brockman et al., 2016).”