Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective

By Kim M. Hazelwood, Sarah Bird, David M. Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, M. Fawzy, Bill Jia, Yangqing Jia and 8 more. 2018

626 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Evaluation & Benchmarks, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Evaluation & Benchmarks

In ML Systems & Hardware

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, methodology): “At-Scale Efficiency Optimization for Facebook Data Centers: Servers in Facebook data center fleets are customized for internal workloads only — machine learning tasks [24] or not [34], [35].”
Measuring the Carbon Intensity of AI in Cloud Instances (2022, background): “This is in fact a common use case in production ML systems deployed at companies, where models are re-trained on a regular schedule to incorporate new data over time [16].”
Chasing Carbon: The Elusive Environmental Footprint of Computing (2020, background): “For instance, Facebook data centers have custom servers for AI inference and training [95].”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “LL has the added beneﬁt of avoiding periodical re-training of models from scratch to learn novel tasks or adapt to new data, with the potential to reduce both computational and energy requirements (Hazelwood et al., 2018; Strubell et al., 2019; Schwartz et al., 2020).”
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing (2019, neutral): “experience [21], [25], [29], [44], [49].”
Simba: Scaling Deep-Learning Inference with Multi-Chip-Module-Based Architecture (2019, background): “its layers running on the Simba system with a batch size of one, as low-latency inference is a highly critical deployment scenario for data center inferencing [32, 39, 43].”
Data-centric Artificial Intelligence: A Survey (2023, positive): “Many AI-driven companies like Meta [90] have established data infrastructure to reliably and efficiently supply data for building AI systems.”
Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures (2018, methodology): “1Facebook currently relies heavily on CPUs for machine learning inference [10].”
SPINN: synergistic progressive inference of neural networks over device and cloud (2020, methodology): “tions. 2 BACKGROUND AND RELATED WORK To optimise the execution of CNN workloads, several solutions have been proposed, from compiler [1, 30, 65] and runtime optimisations [36, 43, 49] to custom cloud [7, 19, 34] and accelerator designs [75, 79]. While these works target a single model with 1AWS Spot”
DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, methodology): “Recall that production datacenters run a variety of recommendation models that evolve over time across heterogeneous hardware platforms with varying SLA targets [9].”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, background): “Once deployed, industry-scale production models may be continuously retrained on new data for potentially many weeks [14] at a time.”
The Architectural Implications of Facebook's DNN-Based Personalized Recommendation (2019, methodology): “Finally, we note that in this paper the open-source DLRM benchmark is used to study the performance of recommendation workloads on server class CPUs.”
3LC: Lightweight and Effective Traffic Compression for Distributed Machine Learning (2018, background): “, 50 Gbps) can be insufficient for multi-tenant distributed training (Hazelwood et al., 2018).”
Heterogeneity-Aware Asynchronous Decentralized Training (2019, background): “It is common that training a model may take hours or even days [17].”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, methodology): “presented the major products and services leveraging machine learning that run on Facebook customized datacenter infrastructure [1].”
Compute Trends Across Three Eras of Machine Learning (2022, background): “The hardware constraints involved in massive training runs (including memory limitations and communication bandwidths) force users to segment massive models into groups of layers, which are then trained in parallel (Hazelwood et al., 2018; Huang et al., 2019; Athlur et al., 2021).”
Benchmarking TPU, GPU, and CPU Platforms for Deep Learning (2019, background): “For example, Facebook reports using dual-socket, high-memory CPU servers to train ranking models for News Feed and to perform anomaly detection (Sigma), both of which are fully-connected networks [20].”
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “With the proliferation of machine learning (ML), model inferences are now not only commonplace but increasingly on the critical path of web requests [29,71].”