Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads

By D. Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, Amar Phanishayee, Matei A. Zaharia. 2020

300 citations recorded by the source corpus.

Specialized accelerators such as GPUs, TPUs, FPGAs, and custom ASICs have been increasingly deployed to train deep learning models. These accelerators exhibit heterogeneous performance behavior across model architectures. Existing schedulers for clusters of accelerators, which are used to arbitrate these expensive training resources across many users, have shown how to optimize for various multi-job, multi-user objectives, like fairness and makespan. Unfortunately, existing schedulers largely do not consider performance heterogeneity. In this paper, we propose Gavel, a heterogeneity-aware scheduler that systematically generalizes a wide range of existing scheduling policies. Gavel expresses these policies as optimization problems, making it easy to optimize for objectives in a heterogeneity-aware way, while also being cognizant of performance optimizations like space sharing. Gavel then uses a round-based scheduling mechanism to ensure jobs receive their ideal allocation given the target scheduling policy. Gavel's heterogeneity-aware policies allow a heterogeneous cluster to sustain higher input load, and improve end objectives such as average job completion time and makespan by up to 3.5x compared to heterogeneity-agnostic policies.

Subfields: Artificial Intelligence, Hardware and Architecture, Information Systems, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

In Information Systems

In ML Systems & Hardware

Citations in the corpus

Pathways: Asynchronous Distributed Dataflow for ML (2022, background): “Cluster schedulers optimize for heterogeneity of ML work-loads (Narayanan et al., 2020) and multi-job, multi-user fairness and performance (Xiao et al., 2018; Ren et al., 2015; Mahajan et al., 2020; Jeon et al., 2018), but resources are still exclusively dedicated to single jobs at long time scales…”
Wavelet: Efficient DNN Training with Tick-Tock Scheduling (2021, unknown): “For the case of GPU heterogeneity, we can wisely balance workload to make sure that all GPUs can ﬁnish a training iteration using the same amount of time (Narayanan et al., 2020) or conduct job migration to form homogeneous execution environments (Xiao et al., 2018).”
Scaling Distributed Machine Learning with In-Network Aggregation (2019, unknown): “While it is hard to predict future evolution of GPU speed vs. network bandwidth, we reason that this scaling factor currently corresponds to the span of two to three GPU generations (the A100 benchmarks at 4.2 × the V100 [75], which in turn is 1.4-2.2 × faster than our P100 [97]) and represents a li”