Effective Straggler Mitigation: Attack of the Clones

By Ganesh Ananthanarayanan, A. Ghodsi, Scott Shenker, Ion Stoica. 2013

554 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Decentralized gradient methods: does topology matter? (2020, background): “Stragglers can significantly reduce computation speed in a multi-machine setting (Ananthanarayanan et al., 2013; Karakus et al., 2017; Li et al., 2018).”
Robust and Communication-Efficient Collaborative Learning (2019, background): “A common approach to mitigate stragglers is to replicate the computing task of the slow nodes to other computing nodes (Ananthanarayanan et al., 2013; Wang et al., 2014), but this is clearly not feasible in collaborative learning.”
Speeding Up Distributed Machine Learning Using Codes (2015, background): “Recently, replication-based approaches have been explored to tackle the straggler problem: by replicating tasks and scheduling the replicas, the runtime of distributed algorithms can be significantly improved [30]–[36].”
Straggler Mitigation in Distributed Optimization Through Data Encoding (2017, methodology): “Use of data replication to aid with the straggler problem has been proposed and studied in [22, 1], and references therein.”
Making Sense of Performance in Data Analytics Frameworks (2015, methodology): “This has driven work on mitigation using task speculation [8] [10] [11] [52] or running shorter tasks to improve load balancing [39].”
Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning (2018, background): “Such techniques have been proposed and analyzed in Gardner et al. (2015); Ananthanarayanan et al. (2013); Shah et al. (2016); Wang et al. (2015); Yadwadkar et al. (2016), among others.”
The Role of Network Topology for Distributed Machine Learning (2019, background): “This requirement makes the system particularly sensitive to stragglers, which can significantly reduce computation speed of a distributed system [11], [12], [13].”
Addressing the straggler problem for iterative convergent parallel ML (2016, methodology): “Stragglers can occur for a number of reasons [5, 6], including hardware heterogeneity [32, 45, 49], hardware failures [6], unbalanced data distribution among tasks, garbage collection in high-level languages, and various OS effects [8, 42].”
Low Latency Geo-distributed Data Analytics (2015, background): “These systems improve data locality of input tasks and fairness [37, 58], and minimize outliers in task execution [16, 19, 61].”
Rateless codes for near-perfect load balancing in distributed matrix-vector multiplication (2018, background): “, as observed in the systems literature [6], [8].”
Approximate Gradient Coding via Sparse Random Graphs (2017, background): “These approaches include replicating jobs across redundant nodes [22] and dropping stragglers in the case that the underlying computation is robust to errors [1, 24].”
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “An example of this contagiousness of unpredictability, known as the “straggler” problem in data analytics frameworks [7, 56], is when a worker executes a request that takes unusually long and the other requests that were enqueued on the worker in the meantime then incur the extra delay from the unex”
Gradient Coding Using the Stochastic Block Model (2018, background): “Current approaches include replicating jobs across nodes [3] and dropping stragglers in settings where the system can tolerate errors [4].”
Communication-Computation Efficient Gradient Coding (2018, methodology): “The most popular ways to introduce redundancy in computation are based on either replication schemes or coding theoretic techniques (Ananthanarayanan et al., 2013; Wang et al., 2014; Shah et al., 2016; Lee et al., 2016).”
Quasar: resource-efficient and QoS-aware cluster management (2014, background): “These straggling tasks are typically identified and relaunched by the framework to ensure timely job completion [1, 3, 4, 17, 23, 37, 70].”
Efficient Memory Disaggregation with Infiniswap (2017, background): “that the higher probabilities in the Google cluster are due to its long-running services, whereas the Facebook cluster runs data analytics with many short tasks [24].”
Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale (2015, background): “This overview highlights the lack of coordination that currently exists between scheduling and straggler mitigation strategies such as speculation.”