Omega: flexible, scalable schedulers for large compute clusters

By Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, John Wilkes. European Conference on Computer Systems, 2013

789 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Hardware and Architecture, Information Systems

View Paper: https://www.semanticscholar.org/paper/1407f94cfed1db0af188ec79b385c7f11ef0aac1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Hardware and Architecture

In Information Systems

Citations in the corpus

Quasar: resource-efficient and QoS-aware cluster management (2014, background): “Finally, most cloud facilities are large and involve thousands of servers and workloads, putting tight constraints on the complexity and time that can be spent making decisions [54].”
Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads (2019, background): “The auction allocates each GPU to exactly one app, thus avoiding any form of conflict resolution or concurrency control [22].”
Occupy the cloud: distributed computing for the 99% (2017, background): “Scalable Scheduling: A number of cluster scheduling papers [40, 32, 31, 21] have looked at providing low latency scheduling for data parallel frameworks running on servers.”
Computation Offloading Toward Edge Computing (2019, background): “Unlike the cloud that has the global view of overall resources and thus usually adopts centralized scheduling policies [115], [116], cloudlets are geographically distributed without a single centralized node, leading to difficulty in performing load balancing [117].”
Resource Management in Clouds: Survey and Research Challenges (2015, neutral): “[191].”
Who Limits the Resource Efficiency of My Datacenter: An Analysis of Alibaba Datacenter Traces (2019, background): “Share state CMSes like Omega [26] allows multiple schedulers have a view of the entire cluster.”
Flexible Scheduling of Distributed Analytic Applications (2016, methodology): “We evaluate our algorithm using an event-based, trace-driven discrete simulator developed to study the scheduler Omega [9], which we extended in order to make it work with applications, instead of low-level jobs and to use the concept of component classes.”
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads (2020, background): “Omega [50], Apollo [16], and Hydra [20] are schedulers that take into account the fact that the target workload shows heterogeneity in the number and duration of constituent tasks.”
Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices (2019, background): “Omega on the other hand, follows a shared-state approach, where multiple concurrent schedulers can view the whole cluster state, with conflicts being resolved through a transactional mechanism [82].”
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads (2019, background): “In contrast, in big data analytics, job execution times range from only tens of milliseconds to a few hours [12,36,40].”
FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices (2020, background): “This increased popularity of service-oriented architectures (SOA) of web services has been made possible by the rise of containerization [21, 70, 92, 108] and container-orchestration frameworks [19, 20, 90, 119] that enable modular, low-overhead, low-cost, elastic, and highefficiency development and”
Splitwise: Efficient Generative LLM Inference Using Phase Splitting (2023, background): “Insights from prior work on partitioned or replicated scheduling could help improve scalability [27], [61], [72] and are orthogonal to Splitwise.”
Multi-tenant GPU Clusters for Deep Learning Workloads: Analysis and Implications (2018, background): “In contrast, in big data analytics, job execution times range from only tens of milliseconds to a few hours [12, 36, 40].”