The Arcade Learning Environment: An Evaluation Platform for General Agents
By Marc G. Bellemare, Yavar Naddaf, Joel Veness, Michael H. Bowling. 2012
3,357 citations recorded by the source corpus.
In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.
Subfields: Artificial Intelligence, Reinforcement Learning, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Pieter Abbeel
- Mohit Bansal
- Ruslan Salakhutdinov
- Quoc V. Le
- Jianfeng Gao
- Percy Liang
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Shimon Whiteson
- Yuxi Li
- Ofir Nachum
- Shixiang Shane Gu
- Marc G. Bellemare
In Evaluation & Benchmarks
- Sergey Levine
- Pieter Abbeel
- Jitendra Malik
- Trevor Darrell
- Abhinav Kumar Gupta
- Joelle Pineau
- Mohit Bansal
- Alexei A. Efros
Citations in the corpus
- Deep AutoRegressive Networks (2013, methodology): “We recorded 100 , 000 frames from five different Atari 2600 games (Bellemare et al., 2013) from random play, recording each frame with 1% probability.”
- Playing Atari with Deep Reinforcement Learning (2013, methodology): “Since our evaluation metric, as suggested by [3], is the total reward the agent collects in an episode or game averaged over a number of games, we periodically compute it during training.”
- Trust Region Policy Optimization (2015, methodology): “tially from run to run (with different random initializations of the policy), but we could not obtain error statistics due to time constraints. made available through the Arcade Learning Environment (Bellemare et al., 2013) The images were preprocessed following the protocol in Mnih et al (2013), an”
- Measuring Massive Multitask Language Understanding (2020, background): “There are 57 tasks in total, which is also the number of Atari games (Bellemare et al., 2013), all of which are listed in Appendix B.”
- Deep Reinforcement Learning from Human Preferences (2017, methodology): “We interface with Mu-JoCo (Todorov et al., 2012) and the Arcade Learning Environment (Bellemare et al., 2013) through the OpenAI Gym (Brockman et al., 2016).”
- Overcoming catastrophic forgetting in neural networks (2016, background): “We asked whether Deep Q Networks (DQNs)—an architecture that has achieved impressive successes in such challenging RL settings (25)—could be harnessed with EWC to successfully support continual learning in the classic Atari 2600 task set (26).”
- Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks (2016, methodology): “Next we apply weight normalization to the problem of Reinforcement Learning for playing games on the Atari Learning Environment [2].”
- ShapeWorld - A new test methodology for multimodal language understanding (2017, methodology): “…platforms following this paradigm have been released, mostly aimed at reinforcement learning: the Ar-cade Learning Environment / Atari 2600 games (Bellemare et al., 2013), OpenAI Gym (Brockman et al., 2016), DeepMind Lab (Beattie et al., 2016), Project Malmo (Johnson et al., 2016), to name a few…”
- Sample Efficient Actor-Critic with Experience Replay (2016, methodology): “…simulated environments, where agents can be trained to learn a large repertoire of cognitive skills, are at the core of recent breakthroughs in AI (Bellemare et al., 2013; Mnih et al., 2015; Schulman et al., 2015a; Narasimhan et al., 2015; Mnih et al., 2016; Brockman et al., 2016; Oh et al.,…”
- OpenAI Gym (2016, methodology): “A variety of benchmarks have been released, such as the Arcade Learning Environment (ALE) [5], which exposed a collection of Atari 2600 games as reinforcement learning problems, and recently the RLLab benchmark for continuous control [6], to which we refer the reader for a survey on other RL…”
- Benchmarking Deep Reinforcement Learning for Continuous Control (2016, background): “Along with this recent progress, the Arcade Learning Environment (ALE) (Bellemare et al., 2013) has become a popular benchmark for evaluating algorithms designed for tasks with high-dimensional state inputs and discrete actions.”
- Reward learning from human preferences and demonstrations in Atari (2018, methodology): “We evaluate our method on the Arcade Learning Environment (Bellemare et al., 2013) because Atari games are RL problems difficult enough to benefit from nonlinear function approximation and currently among the most diverse environments for RL.”
- Decision Transformer: Reinforcement Learning via Sequence Modeling (2021, methodology): “We evaluate on both discrete (Atari (Bellemare et al., 2013)) and continuous (OpenAI Gym (Brockman et al.”
- Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, background): “[32] Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling.”
- The Reactor: A Sample-Efficient Actor-Critic Architecture (2017, background): “In our Elo evaluation difference in ranking 400 correspond to the odds of winning of 10:1. et al., 2013).”