The Architectural Implications of Facebook's DNN-Based Personalized Recommendation

By Udit Gupta, Xiaodong Wang, Maxim Naumov, Carole-Jean Wu, Brandon Reagen, David M. Brooks, Bradford Cottel, Kim M. Hazelwood, Bill Jia, Hsien-Hsin S. Lee and 5 more. 2019

330 citations recorded by the source corpus.

The widespread application of deep learning has changed the landscape of computation in data centers. In particular, personalized recommendation for content ranking is now largely accomplished using deep neural networks. However, despite their importance and the amount of compute cycles they consume, relatively little research attention has been devoted to recommendation systems. To facilitate research and advance the understanding of these workloads, this paper presents a set of real-world, production-scale DNNs for personalized recommendation coupled with relevant performance metrics for evaluation. In addition to releasing a set of open-source workloads, we conduct in-depth analysis that underpins future system design and optimization for at-scale recommendation: Inference latency varies by 60% across three Intel server generations, batching and co-location of inference jobs can drastically improve latency-bounded throughput, and diversity across recommendation models leads to different optimization strategies.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, Recommender Systems, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In Recommender Systems

In ML Systems & Hardware

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “For a number of important recommendation and ranking use cases, the embedding operation dominates the inference execution time [26], [32].”
Chasing Carbon: The Elusive Environmental Footprint of Computing (2020, background): “It represents a data-center-scale server [79], [80], yielding a 2.6 × greater manufacturing carbon footprint.”
RecSSD: near data processing for solid state drive based recommendation inference (2021, methodology): “fact, in many cases, the size of recommendation models is set by the amount of memory available on servers [20].”
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems (2020, background): “Each node processes the entire mini-batch of data and communicates the activations forward or error gradients backwards to other nodes.”
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale (2020, background): “In addition, recent studies have started analyzing the system- and architecturelevel implications of neural recommendation inference [20], [21], [24].”
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing (2019, methodology): “Recommendation systems employ three levels of parallelism, shown in Figure 3, to achieve high throughput under strict latency constraints [75].”
High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (2021, background): “At Facebookwe recommendation models extensively for ranking and click through rate (CTR) prediction, including News Feed and search services [12, 13, 38, 41], and is the single largest AI application in terms of infrastructure demand in its data-centers.”
Understanding Capacity-Driven Scale-Out Neural Recommendation Inference (2020, methodology): “However, more recent recommender systems have used deep neural networks to combine a variety of dense and sparse input features into a more generalized predictive model [1], [3], [26]–[29].”
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models (2021, methodology): “To understand Rec-TT training performance for other categories of DLRMs that are embedding-dominated (Gupta et al., 2019; Gupta et al., 2020; Ke et al., 2020), we develop a suite of microbenchmarks to synthetically generate embedding vectors with a polling factor (P) of 10 and 100.”
MLPerf Inference Benchmark (2019, background): “, Transformers [50], BERT [16], and DLRM [38], [19]), but we aim to add them soon.”
This paper is included in the Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation. (background): “…Alibaba) for recommending items to purchase [30,33], social media platforms (e.g. Facebook, Twitter) for providing the most relevant content [14], enter-tainment services (e.g. Netﬂix, Youtube) for promoting new playlists [7,12], and storage services (e.g. Google Drive) for enabling quick…”
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale (2021, methodology): “Among all the deep learning applications, recent work showed, in 2019, more than 70% of the total machine learning inference cycles served by Facebook’s datacenter ﬂeets are devoted to recommendation use cases [24].”
Software-hardware co-design for fast and scalable training of deep learning recommendation models (2021, methodology): “At Meta, we have been using recommendation models extensively for ranking and click through rate (CTR) prediction, including news feed and search services [15, 17, 42, 47].”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, methodology): “This is due to their use of embedding layers which demand ordersof-magnitude higher memory capacity and exhibit significantly lower compute-intensity [11, 26, 33].”
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery (2020, background): “Across clusters ser-vicing machine learning workloads, about 50% of training and 80% of inference cycles are dedicated to recommendation models at Facebook (Gupta et al., 2020).”