Occupy the cloud: distributed computing for the 99%

By Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, B. Recht. ACM Symposium on Cloud Computing, 2017

526 citations recorded by the source corpus.

Distributed computing remains inaccessible to a large number of users, in spite of many open source platforms and extensive commercial offerings. While distributed computation frameworks have moved beyond a simple map-reduce model, many users are still left to struggle with complex cluster management and configuration tools, even for running simple embarrassingly parallel jobs. We argue that stateless functions represent a viable platform for these users, eliminating cluster management overhead, fulfilling the promise of elasticity. Furthermore, using our prototype implementation, PyWren, we show that this model is general enough to implement a number of distributed computing models, such as BSP, efficiently. Extrapolating from recent trends in network bandwidth and the advent of disaggregated storage, we suggest that stateless functions are a natural fit for data processing in future computing environments.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/a17141d9184ff8e05d90a8073ee9f0bef48bce71

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

FaasCache: keeping serverless computing alive with greedy-dual caching (2021, methodology): “Functions as a Service (FaaS) is an emerging and popular cloud computing model, where applications use cloud resources through user defined łfunctionsž that execute application code [38, 39, 53].”
Cloud Programming Simplified: A Berkeley View on Serverless Computing (2019, background): “First, managing clusters is a big barrier for many non-CS scientists [27].”
SOCK: Rapid Task Provisioning with Serverless-Optimized Containers (2018, methodology): “Prior studies have ported various applications to the lambda model in order to evaluate platform performance [21, 33].”
What serverless computing is and should become (2021, background): “Simplified distributed systems programming is a core benefit of serverless computing, 18 and while much previous work in this area is relevant, the serverless setting calls for a new perspective and adds urgency.”
The rise of serverless computing (2019, background): “From a cost perspective, the benefits of a serverless architecture are most apparent for bursty [5,6,30], compute intensive [7,8] workloads.”
Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads (2017, methodology): “-point computational power for an embarrassingly parallel job (after PyWren [14]). mu starts 3,600 workers in less than 3 seconds.”
Next generation cloud computing: New trends and research directions (2017, background): “As the name implies ‘serverless’ does not mean that computing will be facilitated without servers [57,58].”
Pocket: Elastic Ephemeral Storage for Serverless Analytics (2018, methodology): “MapReduce Sort: We implement a MapReduce sort application on AWS Lambda, similar to PyWren [45].”
Serverless Computing: One Step Forward, Two Steps Back (2018, methodology): “The PyWren [13] and ExCamera [6] research projects have shown that AWS Lambda can be made (with some effort) to perform a wider variety of such “map” functions, including some simple featurization and linear algebra workloads.”
Peeking Behind the Curtains of Serverless Platforms (2018, background): “Now it is used as a simple programming model for a variety of applications [14,22,42].”
SAND: Towards High-Performance Serverless Computing (2018, methodology): “Despite its recency, serverless computing has already been used in various scenarios including Internet of Things and edge computing [7,16], parallel data processing [33, 34], data management [14], system security enhancement [13], and low-latency video processing [20].”