Reining in the Outliers in Map-Reduce Clusters using Mantri

By Ganesh Ananthanarayanan, Srikanth Kandula, Albert G. Greenberg, Ion Stoica, Yi Lu, Bikas Saha, E. Harris. 2010

809 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Robust and Communication-Efficient Collaborative Learning (2019, background): “The straggler problem has been widely observed in distributed computing clusters (Dean and Barroso, 2013; Ananthanarayanan et al., 2010).”
Speeding Up Distributed Machine Learning Using Codes (2015, background): “[26] characterize the impact and causes of stragglers that arise due to resource contention, disk failures, varying network conditions, and imbalanced workload.”
Ray: A Distributed Framework for Emerging AI Applications (2017, background): “he case of a robot interacting with the physical environment, we need to infer the state of 2A stage is the unit of parallelism in BSP. 3Many straggler mitigation techniques depend on this assumption [8]. the environment and compute a new action in a matter of milliseconds. Similarly, simulations mi”
Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing (2014, background): “For example, many existing works on outlier detection and straggler mitigation (e.g., LATE [30], Mantri [4], and Jockey [9]) are complementary to our work and can be integrated with the Apollo framework for reducing job latency.”
Making Sense of Performance in Data Analytics Frameworks (2015, methodology): “This has driven work on mitigation using task speculation [8] [10] [11] [52] or running shorter tasks to improve load balancing [39].”
Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics (2016, background): “The problem of dealing with stragglers, or tasks which take much longer than other tasks is one of the main challenges in large scale data analytics [80, 13, 33].”
Jockey: guaranteed job latency in data parallel clusters (2012, background): “Additional control knobs such as the aggressiveness of mitigating stragglers [2], the OS priority of tasks, and the bandwidth shares of network transfers, could broaden what Jockey can do to meet SLOs.”
Towards predictable datacenter networks (2011, background): “Categories and Subject Descriptors: C.2.3 [ComputerCommunication Networks]: Network Operations General Terms: Algorithms, Design, Performance Keywords: Datacenter, Allocation, Virtual Network, Bandwidth”
SkewTune: mitigating skew in mapreduce applications (2012, methodology): “Finally, resource-aware scheduling techniques [1] are complementary to SkewTune.”
Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads (2012, background): “The dominance of small jobs complicates efforts to rein in stragglers [12], tasks that execute significantly slower than other tasks in a job and delay job completion.”
ARIA: automatic resource inference and allocation for mapreduce environments (2011, background): “Much of the recent work also focuses on anomaly detection, stragglers and outliers control in MapReduce environments [11, 18, 1] as well as on optimization and tuning cluster parameters and testbed configuration [6, 9].”
Using Straggler Replication to Reduce Latency in Large-scale Parallel Computing (2015, background): “A line of systems work [2, 3, 15, 23] further developed this idea to handle various performance variability issues in data centers.”
Near-Optimal Straggler Mitigation for Distributed Gradient Methods (2017, background): “For example, it was shown in [5] that over a wide range of production jobs, stragglers can prolong the completion time by 34% at median.”
Split/Merge: System Support for Elastic Execution in Virtual Middleboxes (2013, neutral): “For example, it is their responsibility to manage replicas and ensure that each replica will be assigned the same amount of work [1].”
Resource Management with Deep Reinforcement Learning (2016, background): “Several scheduler designs address speciﬁc issues or objectives, such as fairness [41, 16], locality [22, 41], packing [17] and strag-gling tasks [5].”