Workload analysis of a large-scale key-value store

By Berk Atikoglu, Yuehai Xu, E. Frachtenberg, Song Jiang, Mike Paleczny. Measurement and Modeling of Computer Systems, 2012

1,151 citations recorded by the source corpus.

Key-value stores are a vital component in many scale-out enterprises, including social networks, online retail, and risk analysis. Accordingly, they are receiving increased attention from the research community in an effort to improve their performance, scalability, reliability, cost, and power consumption. To be effective, such efforts require a detailed understanding of realistic key-value workloads. And yet little is known about these workloads outside of the companies that operate them. This paper aims to address this gap. To this end, we have collected detailed traces from Facebook's Memcached deployment, arguably the world's largest. The traces capture over 284 billion requests from five different Memcached use cases over several days. We analyze the workloads from multiple angles, including: request composition, size, and rate; cache efficacy; temporal patterns; and application use cases. We also propose a simple model of the most representative trace to enable the generation of more realistic synthetic workloads by the community. Our analysis details many characteristics of the caching workload. It also reveals a number of surprises: a GET/SET ratio of 30:1 that is higher than assumed in the literature; some applications of Memcached behave more like persistent storage than a cache; strong locality metrics, such as keys accessed many millions of times a day, do not always suffice for a high hit rate; and there is still room for efficiency and hit rate improvements in Memcached's implementation. Toward the last point, we make several suggestions that address the exposed deficiencies.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/793fc9ff28c3b2c84d2bf07b2c2cf85a51c188bb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Stronger Semantics for Low-Latency Geo-Replicated Storage (2013, neutral): “When only some writes are transactional and when the writes are a minority of system operations (as found in prior studies [7, 28]), this overhead has a”
Shenango: Achieving High CPU Efficiency for Latency-sensitive Datacenter Workloads (2019, methodology): “We use the USR workload from [13]: requests follow a Poisson arrival process and consist of 99.”
Using RDMA efficiently for key-value services (2014, methodology): “We chose this item size because it is representative of real-life workloads: an analysis of Facebook’s general-purpose key-value store [6] showed that the 50-th percentile of key sizes is approximately 30 bytes, and that of value sizes is 20 bytes.”
Datacenter RPCs can be General and Fast (2018, background): “, when messages are small [15, 50], the network is congestion-free, and RPC handlers are short.”
NetCache: Balancing Key-Value Stores with Fast In-Network Caching (2017, methodology): “99), which are typical workloads for testing key-value stores [28, 37] and are evidenced by real-world deployments [2, 11].”
Efficient Memory Disaggregation with Infiniswap (2017, methodology): “We use memaslap , a load generation and benchmarking tool for Memcached, to measure performance using recent data published by Facebook [26].”
Characterizing, Modeling, and Benchmarking RocksDB Key-Value Workloads at Facebook (2020, neutral): “analyzed the KV workloads of the large-scale Memcached KV-store at Facebook [5].”
Scaling Memcache at Facebook (2013, neutral): “[8] for a more detailed analysis of our workload.”
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark (2018, methodology): “Researchers have developed many methods for benchmarking computer systems and HPC systems [9,16,27].”
Inside the Social Network's (Datacenter) Network (2015, background): “…trafﬁc patterns exhibited by Facebook’s Hadoop deployments comport well with those reported in the literature, signiﬁcant portions of Facebook’s service architecture [10, 15] vary dramatically from the MapReduce-style infrastructures studied previously, leading to vastly different trafﬁc patterns.”
WiscKey (2016, background): “…et al. 2010; Andersen et al. 2009; Atikoglu et al. 2015; Debnath et al. 2010; Lim et al. 2011], though value sizes can vary widely (e.g., 100B to larger than 4KB) [Ahn et al. 2016; Atikoglu et al. 2015; Debnath et al. 2010; Golan-Gueta et al. 2015; Lai et al. 2015; Sears and Ramakrishnan 2012].”
Flat combining and the synchronization-parallelism tradeoff (2010, background): “As reported in a study of Facebook’s KV cache workloads on its production Memcached system, most keys have a size between 20 to 40 bytes [1], which makes trie an undesirable choice.”
IncBricks: Toward In-Network Computation with an In-Network Cache (2017, methodology): “Based on previous studies [15, 40], the workload is conﬁg-ured with (1) a Zipﬁan distribution [3] of skewness 0.99; (2) 95%/5% read/write ratio; (3) 1 million keys with 1024B value size; (4) two deployments: within-cluster and cross-cluster.”
pFabric: minimal near-optimal datacenter transport (2013, background): “Load balancing: Finally, there are a number of proposals on efficient load balancing techniques for datacenter fabrics [2, 17, 22].”
Homa: a receiver-driven low-latency transport protocol using network priorities (2018, neutral): “model of the ETC workload in Section 5 and Table 5 of [5].”
IX: A Protected Dataplane Operating System for High Throughput and Low Latency (2014, background): “Our work focuses on improving operating systems for applications with aggressive networking requirements running on multi-core servers.”
The RAMCloud Storage System (2015, result): “The uniform random case represents a workload with no locality; Zipfian represents locality similar to what has been observed in memcached deployments [Atikoglu et al. 2012].”