Ray: A Distributed Framework for Emerging AI Applications
By Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, William Paul, Michael I. Jordan, Ion Stoica. 2017
1,702 citations recorded by the source corpus.
The next generation of AI applications will continuously interact with the environment and learn from these interactions. These applications impose new and demanding systems requirements, both in terms of performance and flexibility. In this paper, we consider these requirements and present Ray---a distributed system to address them. Ray implements a dynamic task graph computation model that supports both the task-parallel and the actor programming models. To meet the performance requirements of AI applications, we propose an architecture that logically centralizes the system's control state using a sharded storage system and a novel bottom-up distributed scheduler. In our experiments, we demonstrate sub-millisecond remote task latencies and linear throughput scaling beyond 1.8 million tasks per second. We empirically validate that Ray speeds up challenging benchmarks and serves as both a natural and performant fit for an emerging class of reinforcement learning applications and algorithms.
Subfields: Artificial Intelligence, Hardware and Architecture, Information Systems, ML Systems & Hardware
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Quoc V. Le
- Yoshua Bengio
- Mohit Bansal
- Pieter Abbeel
- Jianfeng Gao
- Ruslan Salakhutdinov
- Percy Liang
In Hardware and Architecture
- Ion Stoica
- Anima Anandkumar
- Nando de Freitas
- Ziyun Wang
- Lars Kotthoff
- Tom Gibbs
- Gabriel Barth-Maron
- Dan Belov
In Information Systems
- Ion Stoica
- Joseph E. Gonzalez
- Michael Isard
- Pieter Abbeel
- Yinmin Zhong
- Li Fei-Fei
- Michael I. Jordan
- B. Recht
In ML Systems & Hardware
- Jia Deng
- Alejandro Newell
- Frank Hutter
- Ion Stoica
- Joseph E. Gonzalez
- Michael I. Jordan
- Song Han
- Xin Jin
Citations in the corpus
- Pathways: Asynchronous Distributed Dataflow for ML (2022, methodology): “The performance of Ray and P ATHWAYS are not directly comparable since they use different hardware, but we interpret the re-sults to suggest that, if the full P ATHWAYS design were implemented substituting Ray for P LAQUE , it should be possible to achieve comparable performance.”
- Flower: A Friendly Federated Learning Research Framework (2020, methodology): “In case of limited resources, Ray can sequence the execution of client-side computations, thus enabling a much larger scale of experiments on common hardware.”
- Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules (2019, methodology): “The Ray framework (Moritz et al., 2017) includes a parallelized implementation of PBT ( https://ray. readthedocs.io/en/latest/tune.html ) which handles the exploit-and-explore process in the backend.”
- Fast AutoAugment (2019, methodology): “For scalable implementation, we use Ray (Moritz et al., 2018) library which combines distributed training framework with TPE algorithm.”
- Learning data augmentation policies using augmented random search (2018, methodology): “The data parallel training across multiple GPUs using Ray [13] is exploited to speed up the “child model” training process and collect more reward signals.”
- Acme: A Research Framework for Distributed Reinforcement Learning (2020, methodology): “Fiber (Zhi et al., 2020) and Ray (Moritz et al., 2017) are both generic tools for expressing distributed computations, similar to Launchpad, described below.”
- Simple random search provides a competitive approach to reinforcement learning (2018, methodology): “For example, in their implementation of ES, Moritz et al. [22] allowed each worker to have its own independent estimate of µ j and Σ j .”
- Tune: A Research Platform for Distributed Model Selection and Training (2018, methodology): “To meet these requirements, we propose the Tune user-facing and scheduling APIs (Section 4) and implement it on the Ray distributed computing framework (Moritz et al. (2017)).”
- Effective Diversity in Population-Based Reinforcement Learning (2020, methodology): “All experiments made use of the ray (Moritz et al., 2017) library for parallel computing, with experiments run on a 32-core machine.”
- Qwen-Image Technical Report (2025, neutral): “To ensure both high throughput and training stability when scaling to large-scale GPU clusters, we adopt a Ray (Moritz et al., 2018)-inspired Producer–Consumer framework that decouples data preprocessing from model training.”
- TF-Replicator: Distributed Machine Learning for Researchers (2019, background): “recent frameworks include Mesh-TensorFlow (Shazeer et al., 2018), which defines a new language that simplifies data and model-parallelism by allowing the user to explicitly map sub-computations to mesh-connected hardware; and Ray (Moritz et al., 2018), which unifies simulation, training and serving for”
- Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods (2020, methodology): “Shared memory is used heavily by frameworks like PyTorch (Paszke et al. 2017) and Ray (Moritz et al. 2018).”
- Ray RLLib: A Composable and Scalable Reinforcement Learning Library (2017, methodology): “We demonstrate this principle by building Ray RLlib on top of Ray [40] and show that we can implement a wide range of state-of-the-art algorithms by composing and reusing a handful of standard components.”
- This paper is included in the Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation. (positive): “For the distributed runtime, we use Ray [37] actor to implement the device mesh worker, XLA runtime for executing computation, and NCCL [41] for communication.”
- TÜLU 3: Pushing Frontiers in Open Language Model Post-Training (2024, positive): “In particular, we use Ray [Moritz et al., 2018] to allocate dedicated GPUs to run PagedAttention via vLLM [Kwon et al., 2023].”
- Optuna: A Next-generation Hyperparameter Optimization Framework (2019, positive): “is built on Kubeflow, which is a computing platform for machine learning services that is based on Kubernetes. Tune also supports parallel optimization, and uses the Ray distributed computing platform [11]. However, there are several serious problems that are being overlooked in many of these existin”
- Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, methodology): “The implementation of different inference strategies was done in Tensorflow [Abadi et al., 2016] and the 21 communication between workers is managed using Ray [Moritz et al., 2017].”