RecSSD: near data processing for solid state drive based recommendation inference

By Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David M. Brooks, Gu-Yeon Wei. 2021

135 citations recorded by the source corpus.

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities. Unfortunately, large and fast DRAM-based memories levy high infrastructure costs. Conventional SSD-based storage solutions offer an order of magnitude larger capacity, but have worse read latency and bandwidth, degrading inference performance. RecSSD is a near data processing based SSD memory system customized for neural recommendation inference that reduces end-to-end model inference latency by 2× compared to using COTS SSDs across eight industry-representative models.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In ML Systems & Hardware

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “For a number of important recommendation and ranking use cases, embedding operations dominate model execution time (Gupta et al., 2020; Ke et al., 2020; Wilkening et al., 2021; Sethi et al., 2022).”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, background): “Furthermore, EMBs exhibit irregular memory access patterns [41], and the concurrent vector accesses per-EMB and across EMBs require substantial memory bandwidth [1, 23].”