Heracles: Improving resource efficiency at scale

By David Lo, Liqun Cheng, Rama K. Govindaraju, Parthasarathy Ranganathan, Christos Kozyrakis. International Symposium on Computer Architecture, 2015

598 citations recorded by the source corpus.

User-facing, latency-sensitive services, such as websearch, underutilize their computing resources during daily periods of low traffic. Reusing those resources for other tasks is rarely done in production services since the contention for shared resources can cause latency spikes that violate the service-level objectives of latency-sensitive tasks. The resulting under-utilization hurts both the affordability and energy-efficiency of large-scale datacenters. With technology scaling slowing down, it becomes important to address this opportunity. We present Heracles, a feedback-based controller that enables the safe colocation of best-effort tasks alongside a latency-critical service. Heracles dynamically manages multiple hardware and software isolation mechanisms, such as CPU, memory, and network isolation, to ensure that the latency-sensitive job meets latency targets while maximizing the resources given to best-effort tasks. We evaluate Heracles using production latency-critical and batch workloads from Google and demonstrate average server utilizations of 90% without latency violations across all the load and colocation scenarios that we evaluated.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/1dca94518af97b6e84797b4732e25d105e983682

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices (2019, methodology): “Upon identifying the problematic resource(s), Seer notifies the cluster manager, which takes one of several resource allocation actions, resizing the Docker container, or using mechanisms like Intel’s Cache Allocation Technology (CAT) for last level cache (LLC) partitioning, and the Linux traffic co”
Who Limits the Resource Efficiency of My Datacenter: An Analysis of Alibaba Datacenter Traces (2019, background): “batch-processing applications to harness stranded CPU cycles [19].”
PARTIES: QoS-Aware Resource Partitioning for Multiple Interactive Services (2019, methodology): “Unmanaged results in a dramatic increase in latency, whereas Heracles detects the QoS violation, and pauses all other applications for ive minutes, as speciied in [43].”
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “In a shared setting, preserving predictability becomes more challenging – though not impossible – and this is an active area of research due to a general need to co-locate latency critical datacenter services [42, 47].”
Shenango: Achieving High CPU Efficiency for Latency-sensitive Datacenter Workloads (2019, methodology): “For example, Heracles [48] adjusts CPU isolation mechanisms (e.”