Datacenter-Scale Analysis and Optimization of GPU Machine Learning Workloads

By Lukasz Wesolowski, Bilge Acun, Valentin Andrei, A. Aziz, Gisle Dankel, Chris Gregg, Xiaoqiao Meng, Cyril Meurillon, Denis Sheahan, Lei Tian and 4 more. 2021

22 citations recorded by the source corpus.

In this article, we present a system to collectively optimize efficiency in a very large scale deployment of GPU servers for machine learning workloads at Facebook. Our system 1) measures and stores system-wide efficiency metrics for every executed workflow; 2) aggregates data from across the execution stack to identify optimization opportunities that maximize fleet-wide efficiency improvements; 3) provides periodic and on-demand whole-system profiling for workflows; and 4) automatically analyzes traces for common antipatterns. We present each component of the stack and show case studies demonstrating the use of the tools to significantly improve performance. To our knowledge, our system is the most complete and effective solution for identifying and addressing efficiency problems in datacenter-scale GPU deployments.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, Evaluation & Benchmarks, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In Evaluation & Benchmarks

In ML Systems & Hardware

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, methodology): “However, GPUs can be severely under-utilized during both the ML Experimentation and Training phases (Figure 10) [35].”