Tiresias: A GPU Cluster Manager for Distributed Deep Learning

By Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Harry Liu, Chuanxiong Guo. Symposium on Networked Systems Design and Implementation, 2019

457 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/8d2d560bf1c4c6930d2d1411e48b642bd5b81179

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Zico: Efficient GPU Memory Sharing for Concurrent DNN Training (2021, background): “As the batch size is usually fixed, the computation load and the memory usage characteristic are usually very similar across iterations [15, 43].”
Wavelet: Efficient DNN Training with Tick-Tock Scheduling (2021, background): “Multiple resource scheduling policies have been tailor-made for distributed DNN training jobs recently (Jeon et al., 2019), with focus on fairness (Mahajan et al., 2020), better locality (Xiao et al., 2018), minimizing job completion time (Gu et al., 2019), auto-scaling (Or et al., 2020), etc.…”
Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads (2019, methodology): “Tiresias’s inefficacy arises from its focus on simple resource fairness which ignores placement sensitivity.”
Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications (2019, unknown): “First, the coarse-grained, one-at-a-time GPU allocation model1 hinders the scheduling ability of GPU cluster managers (Gu et al., 2019; Xiao et al., 2018; Hindman et al., 2011; Bernstein, 2014; Vavilapalli et al., 2013; Dutta & Huang, 2019; NVIDIA, 2020b).”
PipeSwitch: Fast Pipelined Context Switching for Deep Learning Applications (2020, background): “To run DNN workloads in a large scale, enterprises build GPU clusters that are either privately [10, 11] or publicly [12–14] shared by multiple users.”
AntMan: Dynamic Scaling on GPU Clusters for Deep Learning (2020, methodology): “Multi-GPU training jobs require gang-scheduling, which means a job will not start training unless all required GPUs are simultaneously available [19, 27].”
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads (2020, methodology): “Thus, on a homogeneous cluster, if a job m with weight w m receives a fraction X m (which is a scalar since there is only one resource type), LAS can be expressed as the following optimization problem: We need to add an additional constraint to ensure that the cluster is not overprovisioned ( ∑ m X ”
A generic communication scheduler for distributed DNN training acceleration (2019, background): “A potential solution is to cooperatively schedule the communication within the shared cluster, and possibly integrate it with DNN cluster job scheduler [17, 29] or Coflow scheduler [13].”
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads (2019, methodology): “Locality awareness improves training time by bringing down the time needed for parameter synchronization [23,50] due to the availability of: (i) fast intra-server interconnects (such as PCIe and NVLink), and (ii) for jobs that do not ﬁt on a single server, high-bandwidth links available within an…”
Nexus: a GPU cluster engine for accelerating DNN-based video analysis (2019, background): “Our mechanisms are also orthogonal to the scheduling, placement, and time-sharing mechanisms in training systems [13, 30, 38] since DNN serving has to be performed within tight latency SLOs while maintaining high utilization.”
DAPPLE: a pipelined data parallel approach for training large models (2020, background): “Some researchers have been seeking for the optimal placement strategy to assign operations in a DNN to different devices[21, 35, 37, 46] to further improve system efficiency.”
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning (2020, background): “Apart from Optimus or Tiresias, any scheduler that does not time-share GPUs such as [9, 25] will also be left with only these two options.”