Stable-Baselines3: Reliable Reinforcement Learning Implementations

By Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, Noah Dormann. 2021

2,998 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization (2022, background): “, 2020) and stable-baselines-3 (Raffin et al., 2021), combining important components from their interfaces.”
Pre-Trained Language Models for Interactive Decision-Making (2022, methodology): “Our implementation is based on Stable Baselines3 [32].”
Gymnasium: A Standard Interface for Reinforcement Learning Environments (2025, background): “Stable Baselines3 (SB3) (Raffin et al., 2021) is a popular library providing a collection of state-of-the-art RL algorithms implemented in PyTorch.”
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments (2023, background): “Deploying an agent on a real robot faces various challenges, such as dealing with real-time control and safety constraints.”
Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning (2025, neutral): “In contrast, traditional RL typically formulates each action distribution as Gaussian with a predefined standard deviation (Schulman et al., 2017; Raffin et al., 2021; Weng et al., 2022), resulting in uniform entropy across actions.”
CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms (2022, methodology): “At the same time, open-source DRL libraries also ﬂourish in the community (Raﬃn et al., 2021; Liang et al., 2018; D’Eramo et al., 2020; Fujita et al., 2021; Weng et al., 2021).”
ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills (2023, methodology): “The Python language determines that the only way to effectively parallelize execution is through multi-process, which is integrated in common visual RL training libraries (Raffin et al., 2021; Weng et al., 2021).”
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents (2024, background): “As has been observed in other domains such as reinforcement learning [17, 40, 8], the current output of foundation models is stochastic and highly sensitive to their inputs [20, 34, 30, 43], motivating the need to test them under varying conditions.”
Reaching the limit in autonomous racing: Optimal control versus reinforcement learning (2023, methodology): “Our customized implementation is based on Stable Baselines3 [38].”
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks (2023, background): “However, because it is written in Lua and does not support the OpenAI gym API, it is difficult to integrate with existing deep learning and DRL libraries (e.g., PyTorch [17] and SB3 [19]).”
Reward Design with Language Models (2023, methodology): “We train DQN agents using the Stable Baselines3 implementation for 500 timesteps with a learning rate of 1 e-4 across 3 seeds (Raffin et al., 2021).”
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study (2024, methodology): “We find three key techniques: (1) advantage normalization (Raffin et al., 2021), (2) large-batch-size training (Yu et al., 2022), and (3) updating the parameters of the reference model with exponential moving average (Ouyang et al., 2022).”
Reinforcement Learning for Language Models (2023, methodology): “We leverage an off-the-shelf implementation of DQN [31] and modify it to perform language generation.”
Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2 (2022, methodology): “The wake agent is based on the Stable Baselines 3 PPO agent (Schulman et al., 2017; Raffin et al., 2021).”