Behaviour Suite for Reinforcement Learning

By Ian Osband, Yotam Doron, Matteo Hessel, John Aslanides, Eren Sezener, Andre Saraiva, Katrina McKinney, Tor Lattimore, Csaba Szepesvari, Satinder Singh and 4 more. 2019

212 citations recorded by the source corpus.

This paper introduces the Behaviour Suite for Reinforcement Learning, or bsuite for short. bsuite is a collection of carefully-designed experiments that investigate core capabilities of reinforcement learning (RL) agents with two objectives. First, to collect clear, informative and scalable problems that capture key issues in the design of general and efficient learning algorithms. Second, to study agent behaviour through their performance on these shared benchmarks. To complement this effort, we open source this http URL, which automates evaluation and analysis of any agent on bsuite. This library facilitates reproducible and accessible research on the core issues in RL, and ultimately the design of superior learning algorithms. Our code is Python, and easy to use within existing projects. We include examples with OpenAI Baselines, Dopamine as well as new reference implementations. Going forward, we hope to incorporate more excellent experiments from the research community, and commit to a periodic review of bsuite from a committee of prominent researchers.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

In Evaluation & Benchmarks

Citations in the corpus

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, background): “Osband et al. (2019) recently proposed a benchmarking suite that disentangles the ability of an algorithm to deal with different types of challenges.”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, methodology): “More recently, DeepMind has released the Behaviour Suite for Reinforcement Learning (bsuite) [154] which is a collection of experiments that investigate capabilities of agents with two objectives: First, to collect the key issues in the design of general and efﬁcient learning algorithms; Second, to…”
Regularized Behavior Value Estimation (2021, methodology): “bsuite (Osband et al., 2019) is a proposed benchmark designed to highlight key aspects of an agent’s scalability such as exploration, memory or credit assignment.”
Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “The Behaviour Suite for Reinforcement Learning (bsuite) (Osband et al., 2020) provides a collection of experiments that investigate the core capabilities of RL agents.”
Leveraging Procedural Generation to Benchmark Reinforcement Learning (2019, background): “Osband et al. (2019) recently proposed seven core RL capabilities to proﬁle with environments in bsuite.”
Mastering diverse control tasks through world models (2025, unknown): “This benchmark includes 23 environments with a total of 468 configurations that are specifically designed to test credit assignment, robustness to reward scale and stochasticity, memory, generalization, and exploration 45 .”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, background): “Our work is conceptually similar to bsuite (Osband et al., 2019), which curates a collection of toy, diagnostic experiments to evaluate different capabilities of a standard, non-continual RL agent.”
An empirical investigation of the challenges of real-world reinforcement learning (2020, methodology): “In a spirit similar to ours, the bsuite framework [Osband et al. 2019] proposes a set of challenges grounded in fundamental problems in RL such as memory, exploration, credit assignment etc.”
Policy Gradients Incorporating the Future (2021, methodology): “We examine the Umbrella-Length task from BSUITE [27], a task involving a long sequential episode where only the first observation (a forecast of rain or shine) and action (whether to take an umbrella) matter, while the rest of the sequence contains random unrelated information.”
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis (2021, methodology): “In a spirit similar to ours, the bsuite framework (Osband et al. 2019) proposes a set of challenges grounded in fundamental problems in RL such as memory, exploration, credit assignment etc.”
Gymnasium: A Standard Interface for Reinforcement Learning Environments (2025, unknown): “It offers a modular design that allows users to easily customize and extend the library’s components.”
Mastering Diverse Domains through World Models (2023, background): “• BSuite This benchmark includes 23 environments with a total of 468 configurations that are specifically designed to test credit assignment, robustness to reward scale and stochasticity, memory, generalization, and exploration 48 .”
On Bonus Based Exploration Methods In The Arcade Learning Environment (2020, background): “Outside of the ALE, Osband et al. (2019) recently provide a collection of experiments that investigate core capabilities of exploration methods.”
The Measure of Intelligence (2019, unknown): “Examples include the Arcade Learning Environment for Reinforcement Learning agents [6], Project MalmÖ [71], the Behavior Suite [68], or the GLUE [95] and SuperGLUE [94] benchmarks for natural language processing.”
Making Sense of Reinforcement Learning and Probabilistic Inference (2020, methodology): “As we highlight this connection, we also clarify some 6DeepSea figure taken with permission from the ‘bsuite’ Osband et al. (2019) potentially confusing details in the popular ‘RL as inference’ framework.”