Understanding Training Efficiency of Deep Learning Recommendation Models at Scale

By Bilge Acun, M. Murphy, Xiaodong Wang, Jade Nie, Carole-Jean Wu, Kim M. Hazelwood. 2020

128 citations recorded by the source corpus.

The use of GPUs has proliferated for machine learning workflows and is now considered mainstream for many deep learning models. Meanwhile, when training state-of-the-art personal recommendation models, which consume the highest number of compute cycles at our large-scale datacenters, the use of GPUs came with various challenges due to having both compute-intensive and memory-intensive components. GPU performance and efficiency of these recommendation models are largely affected by model architecture configurations such as dense and sparse features, MLP dimensions. Furthermore, these models often contain large embedding tables that do not fit into limited GPU memory. The goal of this paper is to explain the intricacies of using GPUs for training recommendation models, factors affecting hardware efficiency at scale, and learnings from a new scale-up GPU server design, Zion.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Optimization & Training

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “However, DNNs, such as neural recommendation models, require signiﬁcantly higher memory capacity and bandwidth [55], [33].”
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models (2021, background): “In response, there is considerable interest to design domainspecific accelerators and at-scale infrastructures (Acun et al., 2021; Alibaba, 2019; Amazon, 2019; Chung et al., 2018; Fowers et al., 2018; Hazelwood et al., 2018; Jouppi et al., 2017; Lee & Rao, 2019; Mattson et al., 2020a; NVIDIA, 2020a;”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, methodology): “Furthermore, EMBs exhibit irregular memory access patterns [41], and the concurrent vector accesses per-EMB and across EMBs require substantial memory bandwidth [1, 23].”
High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (2021, background): “As studied in [1], embedding tables in DLRMs exhibit diverse shapes and dynamic costs.”
Understanding and Co-designing the Data Ingestion Pipeline for Industry-Scale RecSys Training (2021, background): “[8] characterized the implications of recommendation models’ architectures on GPU trainers.”
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery (2020, background): “Thus, the industry devotes significant infrastructure resources to recommendation models—across computing clusters serving a wide variety of machine learning workloads, about 50% of training (Acun et al., 2021) and 80% of inference cycles are dedicated to recommendation at Facebook (Gupta et al.”
Software-hardware co-design for fast and scalable training of deep learning recommendation models (2021, background): “25MQPS)[4, 42], thereby leading to more than 10× lower max achievable effective performance of only 63 TFLOPS/s.”
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning (2021, background): “…inference is becoming more ubiquitous [14,29,41,57,62,65,101,102,119,125,137], performing ML model training in the cloud is still the standard practice for most use cases [1, 29, 43, 54, 81, 88, 101], due to the signiﬁcant computation and memory resource requirements [9,35,69,100,121,122,129,136].”