RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing

By Liu Ke, Udit Gupta, Carole-Jean Wu, B. Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David M. Brooks, Vikas Chandra, Utku Diril and 11 more. 2019

273 citations recorded by the source corpus.

Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a fundamental challenge to accelerate. This paper proposes a lightweight, commodity DRAM compliant, near-memory processing solution to accelerate personalized recommendation inference. The in-depth characterization of production-grade recommendation models shows that embedding operations with high model-, operator- and data-level parallelism lead to memory bandwidth saturation, limiting recommendation inference performance. We propose RecNMP which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models. RecNMP is specifically tailored to production environments with heavy co-location of operators on a single server. Several hardware/software co-optimization techniques such as memory-side caching, table-aware packet scheduling, and hot entry profiling are studied, providing up to $9.8 \times$ memory latency speedup over a highly-optimized baseline. Overall, RecNMP offers $4.2 \times$ throughput improvement and 45.8% memory energy savings.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In Representation & Self-Supervised Learning

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “For a number of important recommendation and ranking use cases, the embedding operation dominates the inference execution time [27], [33].”
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery (2020, background): “…systems tailored to deep learning-based recommendations have also been designed to enable high-performance and energy-efﬁcient execution (Zhao et al., 2020; 2019; Kalamkar et al., 2020; Nvidia, 2019; 2020b; Jouppi et al., 2017; Nvidia, 2020a; Ke et al., 2020; Hwang et al., 2020; Kwon et al., 2019).”
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale (2020, background): “However, neither RecNMP nor TensorDIMM is optimized for gradient aggregation.”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, background): “As a concrete example, for memory-intensive models whose timing composition consists of 35-75% embedding operations [11, 23] (with the remaining time being largely dominated by dense MLP layers and communication), and for which RecShard improves embedding performance by 2.5x, the expected…”
RecSSD: near data processing for solid state drive based recommendation inference (2021, methodology): “Given the low-compute intensity of embedding table operations, recent work also explores the role of near memory processing for Facebook’s recommendation models [26].”
The Architectural Implications of Facebook's DNN-Based Personalized Recommendation (2019, background): “Recent work has also proposed solutions based on near-memory processing to accelerate embedding table operations [41], [42].”
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models (2021, methodology): “To understand Rec-TT training performance for other categories of DLRMs that are embedding-dominated (Gupta et al., 2019; Gupta et al., 2020; Ke et al., 2020), we develop a suite of microbenchmarks to synthetically generate embedding vectors with a polling factor (P) of 10 and 100.”
Understanding Capacity-Driven Scale-Out Neural Recommendation Inference (2020, unknown): “recent works [1]–[5].”