RLlib: Abstractions for Distributed Reinforcement Learning

By Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph E. Gonzalez, Michael I. Jordan, Ion Stoica. 2017

1,043 citations recorded by the source corpus.

Subfields: Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, neutral): “, RLLib (Liang et al., 2018).”
Dopamine: A Research Framework for Deep Reinforcement Learning (2018, background): “RLLab (Duan et al., 2016) emphasizes the benchmarking of continuous control algorithms, while RLLib (Liang et al., 2018) focuses on deﬁning abstractions to optimize reinforcement learning performance in distributed settings.”
Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning (2020, neutral): “All learning was done with RLlib (Liang et al., 2017).”
Gymnasium: A Standard Interface for Reinforcement Learning Environments (2025, background): “Rllib provides a diverse range of algorithms and tools for both single-agent and multi-agent scenarios.”
Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning (2020, methodology): “RLlib (Liang et al., 2018), based on the distributed computation framework Ray (Moritz et al., 2018), and TorchBeast (Küttler et al., 2019) provide optimized implementations of the IMPALA architecture.”
Reverb: A Framework For Experience Replay (2021, background): “Many existing RL libraries and frameworks provide implementations of (Prioritized) Experience Replay; including RLlib (Liang et al., 2018), garage (garage contributors, 2019), stable-baselines (Hill et al., 2018), and Tianshou (Ji-ayi Weng, 2020).”
MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning (2021, methodology): “We conduct experiments on MetaDrive with algorithms mostly implemented in RLLib [25].”
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames (2019, methodology): “Several works have proposed systems for distributed RL (Heess et al., 2017; Liang et al., 2018a; Tian et al., 2019; Silver et al., 2016; OpenAI, 2018; Espeholt et al., 2018).”
HybridFlow: A Flexible and Efficient RLHF Framework (2024, neutral): “However, they only provide primitives for data-parallel training and are constrained to neural networks that are at most hundreds of MB in size [45, 46].”
CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms (2022, background): “At the same time, open-source DRL libraries also ﬂourish in the community (Raﬃn et al., 2021; Liang et al., 2018; D’Eramo et al., 2020; Fujita et al., 2021; Weng et al., 2021).”
Dense reinforcement learning for safety validation of autonomous vehicles (2023, methodology): “In this study, we utilized the PPO algorithm implemented at the RLLib 1.2.0 platform 52 , which was parallelly trained on 500 central-processing-unit cores and 3,500-GB memory high-performance computation cluster at the University of Michigan, Ann Arbor.”
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey (2020, neutral): “Another computational interface, named Flow, enables easy integration of the deep RL library RLlib [157] with SUMO and Aimsun for various control problems in ITS [158].”
PettingZoo: Gym for Multi-Agent Reinforcement Learning (2020, neutral): “…learning libraries or code bases exist (as a rough measure 669 pip-installable packages depend on it at the time of writing GitHub [2021]), only 5 MARL libraries with large user bases exist [Lanctot et al., 2019, Weng et al., 2020, Liang et al., 2018, Samvelyan et al., 2019, Nota, 2020].”
Large Batch Simulation for Deep Reinforcement Learning (2021, methodology): “…observations when operating constantly at maximum speed (0.5 m/s) and capturing 1 frame every 0.25m. of efficient implementations of these ideas have been proposed as part of RL frameworks or in other deep learning libraries (Liang et al., 2018; Stooke & Abbeel, 2019; Küttler et al., 2019).”
A review of cooperative multi-agent deep reinforcement learning (2019, background): “Similarly, Ray platform [142] also recently started to support multi-agent environments, and some of the known algorithms are also added in rllib repository [111, 112].”
Launchpad: A Programming Model for Distributed Machine Learning Research (2021, methodology): “Examples of this include RLgraph and RLlib in the reinforcement learning setting (Liang et al., 2017; Schaarschmidt et al., 2018); Caﬀe and Keras for supervised learning (Jia et al., 2014; Chollet et al., 2015).”