Large-scale cluster management at Google with Borg

By Abhishek Verma, Luis Pedrosa, Madhukar R. Korupolu, David Oppenheimer, Eric Tune, John Wilkes. 2015

1,668 citations recorded by the source corpus.

Google's Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines. It achieves high utilization by combining admission control, efficient task-packing, over-commitment, and machine sharing with process-level performance isolation. It supports high-availability applications with runtime features that minimize fault-recovery time, and scheduling policies that reduce the probability of correlated failures. Borg simplifies life for its users by offering a declarative job specification language, name service integration, real-time job monitoring, and tools to analyze and simulate system behavior. We present a summary of the Borg system architecture and features, important design decisions, a quantitative analysis of some of its policy decisions, and a qualitative examination of lessons learned from a decade of operational experience with it.

Subfields: Computer Networks and Communications, Information Systems, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

  1. Ion Stoica
  2. Mosharaf Chowdhury
  3. Matei A. Zaharia
  4. Christina Delimitrou
  5. Christos Kozyrakis
  6. Hari Balakrishnan
  7. M. Franklin
  8. Raouf Boutaba

In Information Systems

  1. Ion Stoica
  2. Matei A. Zaharia
  3. Mosharaf Chowdhury
  4. Christina Delimitrou
  5. Christos Kozyrakis
  6. M. Franklin
  7. Shivaram Venkataraman
  8. Oriol Vinyals

In ML Systems & Hardware

  1. Ion Stoica
  2. Matei A. Zaharia
  3. M. Franklin
  4. Mosharaf Chowdhury
  5. Derek Gordon Murray
  6. Michael Isard
  7. Martín Abadi
  8. Xin Jin

Citations in the corpus