Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing

By Eric Boutin, Jaliya Ekanayake, Wei Lin, B. Shi, Jingren Zhou, Zhengping Qian, Ming Wu, Lidong Zhou. USENIX Symposium on Operating Systems Design and Implementation, 2014

430 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b6571efa4483aa00d23bbcd36930c4877548ba38

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Splitwise: Efficient Generative LLM Inference Using Phase Splitting (2023, background): “Insights from prior work on partitioned or replicated scheduling could help improve scalability [27], [61], [72] and are orthogonal to Splitwise.”
Twine: A Unified Cluster Management System for Shared Infrastructure (2020, background): “Twine has some similarity to the two-level schedulers (Mesos [17], YARN [37], Apollo [6], and Fuxi [44]), with Twine entitlements as resource offers and Twine scheduler shards as Application Masters (or frameworks in Mesos).”
AntMan: Dynamic Scaling on GPU Clusters for Deep Learning (2020, methodology): “In contrast, data statistic flow information is collected by statistic modules of the local coordinator and aggregated on the cluster statistic module in a bottom-up approach to help make scheduling decisions, which is similar to Apollo [10].”
Multi-tenant GPU Clusters for Deep Learning Workloads: Analysis and Implications (2018, background): “In practice, similar to other scheduling regimes [8, 12, 27, 54], achieving both high locality and low waiting time is still challenging because these two requirements are often at odds with each other.”
Heracles: Improving resource efficiency at scale (2015, background): “Batch analytics frameworks can generate numerous BE tasks and derive significant value even if these tasks are occasionally deferred or restarted [19, 10, 13, 16].”
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads (2019, background): “The effect of locality constraints on queuing delays has been extensively explored in large-scale resource allocation [8,12,27,52].”
Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale (2015, background): “However, as clusters increase in size and complexity, providing scalable and predictable performance is an important ongoing challenge for interactive analytics frameworks [2, 32].”
Gandiva: Introspective Cluster Scheduling for Deep Learning (2018, background): “The low-level mechanisms of Gandiva such as time-slicing and migration also differ significantly from those big data scheduling systems [13, 22, 25, 14], while being surprisingly similar to a traditional operation system [47].”
Low Latency Geo-distributed Data Analytics (2015, background): “2) Reducing data analytics responses: There is a large body of work on improving query response time in data parallel systems [20, 23, 29].”
Large-scale cluster management at Google with Borg (2015, methodology): “Microsoft’s Apollo system [13] uses per-job schedulers for short-lived batch jobs to achieve high throughput on clusters that seem to be comparably-sized to Borg cells.”
Live Video Analytics at Scale with Approximation and Delay-Tolerance (2017, methodology): “It’s the widely-used scheduling policy for cluster computing frameworks like Mesos [49], Yarn [3] and Cosmos [31].”