Making Sense of Performance in Data Analytics Frameworks
By Kay Ousterhout, R. Rasti, Sylvia Ratnasamy, Scott Shenker, Byung-Gon Chun. Symposium on Networked Systems Design and Implementation, 2015
438 citations recorded by the source corpus.
View Paper: https://www.semanticscholar.org/paper/438c798dc62f022efa8916ef27edfc2473fe0031
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Hongzi Mao
- Malte Schwarzkopf
- Derek Gordon Murray
- Mohammad Alizadeh
- Frank McSherry
- Michael Isard
- Ganesh Ananthanarayanan
- Xiaoqi Ren
Citations in the corpus
- Scalability! But at what COST? (2015, background): “We are now starting to see performance studies of the current crop of scalable systems [18], challenging some conventional wisdom underlying their design principles.”
- Big data analytics on Apache Spark (2016, methodology): “Other works compare Apache Spark with other frameworks such as MapReduce [72], study the performance of Apache Spark for specific scenarios such as scale-up configuration [10], analyze the performance of Spark’s programming model for large-scale data analytics [78] and identify the performance bottl”
- Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “An example of this contagiousness of unpredictability, known as the “straggler” problem in data analytics frameworks [6, 46], is when a worker executes a request that takes unusually long and the other requests that were enqueued on the worker in the meantime and then incur the extra delay from the ”
- Learning scheduling algorithms for data processing clusters (2018, background): “ster, our simulator captures several real-world effects: (1) The first “wave” of tasks from a particular stage often runs slower than subsequent tasks. This is due to Spark’s pipelined task execution [63], JIT compilation [47] of task code, and warmup costs (e.g., making TCP connections to other exe”
- Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale (2015, background): “However, despite blacklisting, stragglers occur frequently, often due to intrinsically complex causes such as IO contention, interference by periodic maintenance operations, and hardware behaviors which are hard to model and circumvent [12, 22, 35].”
- Gemini: A Computation-Centric Distributed Graph Processing System (2016, unknown): “Computation, rather than communication, appears to be the actual bottleneck of evaluated distributed systems, which echoes recent findings on distributed data analytics frameworks [37].”