Dominant Resource Fairness: Fair Allocation of Multiple Resource Types

By A. Ghodsi, Matei A. Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, Ion Stoica. Symposium on Networked Systems Design and Implementation, 2011

1,398 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b1a47759dbcf87ed132838be39f8ab244e78e2d6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Multi-resource packing for cluster schedulers (2014, methodology): “Likewise, DRF [12] oers resources to the job whose dominant resource’s allocation is furthest from its fair share.”
Optimus: an efficient dynamic resource scheduler for deep learning clusters (2018, methodology): “Existing schedulers largely adopt FIFO (as in Spark [70]), Dominant Resource Fairness (DRF) [34] (as in Mesos [40] and Yarn [61]) or their variants as default scheduling strategies, which are ignorant of job sizes (represented by input data size, model complexity, or time taken to complete the job).”
Equity of Attention: Amortizing Individual Fairness in Rankings (2018, background): “Fairness dimension has been considered for job dispatching at the OS level, for packet-level network flows [15], for production planning in factories [16], and even for two-sided matchmaking in call centers [3].”
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning (2020, background): “Dominant resource fairness [14] generalizes the notion of max-min fairness to multiple resources, specifically CPU and memory.”
The Unreasonable Fairness of Maximum Nash Welfare (2016, unknown): “The Unreasonable Fairness of Maximum Nash Welfare IOANNIS CARAGIANNIS, University of Patras DAVID KUROKAWA, Carnegie Mellon University HERVÉ MOULIN, University of Glasgow and Higher School of Economics, St Petersburg ARIEL D. PROCACCIA, Carnegie Mellon University NISARG SHAH, Carnegie Mellon…”
Resource Management with Deep Reinforcement Learning (2016, background): “Several scheduler designs address speciﬁc issues or objectives, such as fairness [41, 16], locality [22, 41], packing [17] and strag-gling tasks [5].”
Quasar: resource-efficient and QoS-aware cluster management (2014, background): “Mesos processes these requests and, based on availability and fairness issues [27], makes resource offers to individual frameworks (e.”
Omega: flexible, scalable schedulers for large compute clusters (2013, background): “The allocator avoids con.icts by only offering a given resource to one framework at a time, and attempts to achieve dominant resource fairness (DRF) [11] by choosing the order and the sizes of its offers.3 Because only one framework is examining a resource at a time, it effectively holds a lock”
Mobile Edge Computing (2021, background): “The multi-resource fairness problem has been studied in the ﬁeld of operations research [50], and seminal works in the networking and computing context have appeared in [20].”
this paper (2009, unknown): “We demonstrate the insufficiency of controlling the frame rate with queue states with a feedback algorithm based on current queue length: it observes the current queue length at the arrival of each frame, and maps the queue length of { 0, 1+ } to frame-rate { 60, 24 } fps. • AFR-QWait .”
End-to-end Performance Isolation Through Virtual Datacenters (2014, methodology): “Through a prototype deployed across 113 VMs, three appliances, and a 40 Gbps network, we show that Pulsar enforces tenants’ VDCs while imposing overheads of less than 2% at the data and control plane.”
Live Video Analytics at Scale with Approximation and Delay-Tolerance (2017, background): “Production systems for stream processing like Storm [4], Stream- Scope [62], Flink [2], Trill [36] and Spark Streaming [89] allocate resources among multiple queries only based on resource fairness [7] [10] [27] [43] [51] common to cluster managers like Yarn [3] and Mesos [49].”
Performance Isolation and Fairness for Multi-Tenant Cloud Storage (2012, unknown): “Finally, to manage different resource bottlenecks on contended nodes, we enforce dominant resource fairness [9] between tenants at the node level, while providing maxmin fairness at the service level.”
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads (2020, background): “Mesos and YARN do not reason about interchangeable resource types that can run the same computation: for example, Mesos’s DRF multi-resource sharing policy [26] decides how to give jobs allocations of distinct resource types, such as RAM and CPUs, but assumes that each job has declared which resourc”
Large-scale cluster management at Google with Borg (2015, methodology): “DRF [29, 35, 36, 66] was initially developed for Mesos; Borg uses priorities and admission quotas instead.”
Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center (2011, background): “First, because some frameworks will always reject certain resources, Mesos lets them short-circuit the rejection process and avoid communication by providing filters to the master.”
FairCloud: sharing the network in cloud computing (2012, unknown): “Strategy-proofness: Tenants cannot improve their allocations by lying about their demands [14].”
Learning scheduling algorithms for data processing clusters (2018, methodology): “Indeed, many fair scheduling policies [21, 31] divide resources without paying attention to their decisions’ efficiency: sometimes, an “unfair” schedule results in a more efficient overall execution.”