Scalability! But at what COST?

By Frank McSherry, Michael Isard, Derek Gordon Murray. 2015

368 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022, methodology): “[57] Frank McSherry, Michael Isard, and Derek G Murray.”
TensorFlow: A system for large-scale machine learning (2016, neutral): “Although TensorFlow is a system for “large-scale” machine learning, it is imperative that scalability does not mask poor performance at small scales [49].”
Gemini: A Computation-Centric Distributed Graph Processing System (2016, positive): “To study the performance loss, we profiled several representative graph-parallel systems, including Ligra [47], Galois [36], PowerGraph [16], PowerLyra [12], as well as the optimized single-thread implementation proposed in the COST paper [33] for reference.”
Occupy the cloud: distributed computing for the 99% (2017, background): “Thus, we see that this model is a good fit where a high degree of parallelism is initially required to do ETL / featurization but a single node is sufficient (and most efficient [25]) for model building.”
Semantics-Preserving Parallelization of Stochastic Gradient Descent (2018, background): “As machines with terabytes of memory become commonplace (As of February 2017, one can rent an X1 instance from AWS with 2 TB memory and 128 cores for less than $4 per hour [10]), machine learning tasks on large datasets can be done efficiently on single machines without paying the inherent cost of d”
The Computational Limits of Deep Learning (2020, background): “Similar issues arise in studies of parallelism and lead to perverse conclusions including a parallel algorithm that ‘scales well’ being run on 128 cores and being outperformed by a serial algorithm running on a single core [71].”
Ground: A Data Context Service (2017, background): “Queries on small sub-graphs should be very fast—ideally as fast as an in-memory graph system [23].”
The GAP Benchmark Suite (2015, background): “iverse workload consisting of multiple kernels and input graphs. A key aspect of our benchmark suite is specifying the benchmark itself (this document). Other research eﬀorts have released their code [22, 26, 29, 31], which eases comparisons to themselves, but the evaluator is still responsible for ”
EmptyHeaded: A Relational Engine for Graph Processing (2015, background): “these specialized approaches against traditional OLAP engines is a source of much debate [57], as some researchers believe general approaches can compete with and outperform these specialized designs [20,38]. Recent products, such as SAP HANA, integrate graph accelerators as part of a general OLAP e”
Scientiﬁc Benchmarking of Parallel Computing Systems Twelve ways to tell the masses when reporting performance results (2017, background): “Furthermore, speedup itself is a rather meaningless measure because it will typically be higher on slow processors and / or less optimized codes (e.g., [41]).”