Autopilot: workload autoscaling at Google
By K. Rządca, P. Findeisen, J. Świderski, Przemyslaw Zych, Przemyslaw Broniek, J. Kusmierek, Pawel Krzysztof Nowak, Beata Strack, Piotr Witusowski, S. Hand and 1 more. European Conference on Computer Systems, 2020
354 citations recorded by the source corpus.
In many public and private Cloud systems, users need to specify a limit for the amount of resources (CPU cores and RAM) to provision for their workloads. A job that exceeds its limits might be throttled or killed, resulting in delaying or dropping end-user requests, so human operators naturally err on the side of caution and request a larger limit than the job needs. At scale, this results in massive aggregate resource wastage. To address this, Google uses Autopilot to configure resources automatically, adjusting both the number of concurrent tasks in a job (horizontal scaling) and the CPU/memory limits for individual tasks (vertical scaling). Autopilot walks the same fine line as human operators: its primary goal is to reduce slack - the difference between the limit and the actual resource usage - while minimizing the risk that a task is killed with an out-of-memory (OOM) error or its performance degraded because of CPU throttling. Autopilot uses machine learning algorithms applied to historical data about prior executions of a job, plus a set of finely-tuned heuristics, to walk this line. In practice, Autopiloted jobs have a slack of just 23%, compared with 46% for manually-managed jobs. Additionally, Autopilot reduces the number of jobs severely impacted by OOMs by a factor of 10. Despite its advantages, ensuring that Autopilot was widely adopted took significant effort, including making potential recommendations easily visible to customers who had yet to opt in, automatically migrating certain categories of jobs, and adding support for custom recommenders. At the time of writing, Autopiloted jobs account for over 48% of Google's fleet-wide resource usage.
Subfields: Computer Networks and Communications, Information Systems
View Paper: https://www.semanticscholar.org/paper/f1fe5a8700e7c915a7e58dd69bd0e397f6d0a316
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
In Information Systems
Citations in the corpus
- FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices (2020, methodology): “For instance, Auto-pilot [88] combines time series analysis and RL algorithms to scale the number of containers and associated CPU/RAM.”
- Twine: A Unified Cluster Management System for Shared Infrastructure (2020, background): “Borg’s Autopilot [34] adjusts the CPU and memory allocated to each task–this is an area of future work for Twine.”