Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs

By Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David W. Nellans, S. Keckler. 2019

65 citations recorded by the source corpus.

GPUs accelerate high-throughput applications, which require orders-of-magnitude higher memory bandwidth than traditional CPU-only systems. However, the capacity of such high-bandwidth memory tends to be relatively small. Buddy Compression is an architecture that makes novel use of compression to utilize a larger buddy-memory from the host or disaggregated memory, effectively increasing the memory capacity of the GPU. Buddy Compression splits each compressed 128B memory-entry between the high-bandwidth GPU memory and a slower-but-larger buddy memory such that compressible memory-entries are accessed completely from GPU memory, while incompressible entries source some of their data from off-GPU memory. With Buddy Compression, compressibility changes never result in expensive page movement or re-allocation. Buddy Compression achieves on average $1.9 \times$ effective GPU memory expansion for representative HPC applications and $1.5 \times$ for deep learning training, performing within 2% of an unrealistic system with no memory limit. This makes Buddy Compression attractive for performance-conscious developers that require additional GPU memory capacity.

Subfields: Efficiency & Compression, ML Systems & Hardware

Citations in the corpus

• ZeRO-Offload: Democratizing Billion-Scale Model Training (2021, neutral): “Many efforts have been made in this direction [8, 9, 11, 17, 23, 23, 24, 32–34].”
• Zico: Efficient GPU Memory Sharing for Concurrent DNN Training (2021, unknown): “Many approaches were invented to reduce memory footprint of DNN training, including HW-based compression techniques [7, 30].”