Gandiva: Introspective Cluster Scheduling for Deep Learning

By Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang and 2 more. 2018

605 citations recorded by the source corpus.

Subfields: ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

Citations in the corpus

Pathways: Asynchronous Distributed Dataflow for ML (2022, background): “Cluster schedulers optimize for heterogeneity of ML work-loads (Narayanan et al., 2020) and multi-job, multi-user fairness and performance (Xiao et al., 2018; Ren et al., 2015; Mahajan et al., 2020; Jeon et al., 2018), but resources are still exclusively dedicated to single jobs at long time scales…”
A System for Massively Parallel Hyperparameter Tuning (2018, methodology): “Additionally, in contrast to the user-speciﬁed automated scaling capability for parallel training presented in Xiao et al. (2018), we propose to automate appropriate autoscaling limits by using the performance prediction framework by Qi et al. (2017).”
Multi-tenant GPU Clusters for Deep Learning Workloads: Analysis and Implications (2018, background): “Support for job migration to defragment the cluster [52], especially applied to smaller jobs, will mitigate interference for small jobs, as much as possible, and will support intra-job locality for large jobs.”
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads (2019, methodology): “In particular, while prior work discusses efficiency of distributed training for a certain job size or a configured placement [21, 52], we perform an analysis on the aggregated efficiency for a range of job sizes for the first time.”
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning (2020, methodology): “However, compared to [41], we optimize the migration performance overhead by implementing a three-phase context switch called the suspend-preload-resume.”
A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters (2020, background): “To run many such tasks and achieve high resource utilization, large GPU clusters with thousands or more GPUs are introduced [29, 35, 52, 71].”
Nexus: a GPU cluster engine for accelerating DNN-based video analysis (2019, background): “Our mechanisms are also orthogonal to the scheduling, placement, and time-sharing mechanisms in training systems [13, 30, 38] since DNN serving has to be performed within tight latency SLOs while maintaining high utilization.”
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads (2020, methodology): “Modern DNN schedulers leverage the fact that DNN training is iterative to suspend and resume training at iteration boundaries [28, 58]; this ensures that jobs can be time multiplexed over the existing physical resources.”
Tiresias: A GPU Cluster Manager for Distributed Deep Learning (2019, methodology): “One must employ preemption to avoid head-of-line (HOL) blocking of smaller/shorter jobs by the larger/longer ones – HOL blocking is a known problem of FIFO scheduling currently used in production [41].”
Capuchin: Tensor-based GPU Memory Management for Deep Learning (2020, background): “Although it can reduce GPU memory footprint by decreasing batch size per GPU, recent research [34] has shown that the training pipeline will bound in communication due to frequent model aggregation.”
Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes (2019, background): “Moreover, users often run multiple training tasks to achieve the best result for a specific mission [5].”
Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications (2019, methodology): “Gandiva [50] uses MPS as well, and the authors report unpredictable performance.”
Wavelet: Efficient DNN Training with Tick-Tock Scheduling (2021, methodology): “For the case of GPU heterogeneity, we can wisely balance workload to make sure that all GPUs can ﬁnish a training iteration using the same amount of time (Narayanan et al., 2020) or conduct job migration to form homogeneous execution environments (Xiao et al., 2018).”
PipeSwitch: Fast Pipelined Context Switching for Deep Learning Applications (2020, methodology): “This solution is used by existing systems like Gandiva [24] for task switching, which reported similar second-scale overhead.”
Zico: Efficient GPU Memory Sharing for Concurrent DNN Training (2021, background): “Prior work focuses on either temporally multiplexing GPU in its entirety [26,43,44] or spatially sharing compute units [10].”