An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems

By Yu Gan, Yanqi Zhang, Dailun Cheng, A. Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson and 14 more. International Conference on Architectural Support for Programming Languages and Operating Systems, 2019

832 citations recorded by the source corpus.

Cloud services have recently started undergoing a major shift from monolithic applications, to graphs of hundreds or thousands of loosely-coupled microservices. Microservices fundamentally change a lot of assumptions current cloud systems are designed with, and present both opportunities and challenges when optimizing for quality of service (QoS) and cloud utilization. In this paper we explore the implications microservices have across the cloud system stack. We first present DeathStarBench, a novel, open-source benchmark suite built with microservices that is representative of large end-to-end services, modular and extensible. DeathStarBench includes a social network, a media service, an e-commerce site, a banking system, and IoT applications for coordination control of UAV swarms. We then use DeathStarBench to study the architectural characteristics of microservices, their implications in networking and operating systems, their challenges with respect to cluster management, and their trade-offs in terms of application design and programming frameworks. Finally, we explore the tail at scale effects of microservices in real deployments with hundreds of users, and highlight the increased pressure they put on performance predictability.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/a022334521d88eb0181e76f01b53ce42e7dcc302

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis (2021, methodology): “This result indicates that the scale of existing benchmarks is far smaller than that in real traces [16, 27, 43].”
Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting (2020, methodology): “The results are shown in Figure 13-a. DeathStar represents real-world lightweight serverless functions.”
DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, background): “Furthermore, recent work has opensourced benchmarks for studying the performance implication of at-scale execution of latency critical datacenter workloads and cloud micro-services [26], [29].”
Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices (2019, background): “The end-to-end applications and tracing infrastructure are described in more detail and open-sourced in [45].”