Ultra-Performance Pascal GPU and NVLink Interconnect

By D. Foley, John Danskin. 2017

202 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Pathways: Asynchronous Distributed Dataflow for ML (2022, background): “GPUs use interconnects such as NVLink for high-speed communication between “islands” of accelerators on a small number of hosts (Naumov et al., 2020), and use RDMA capabilities of ethernet and Inﬁniband NICs (GPUDirect) to rapidly communicate between the islands.”
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs (2019, methodology): “[84] D.”
Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect (2019, methodology): “reported to be twice as efficient as PCIe [12].”
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems (2020, methodology): “To illustrate the performance advantages of using GDR, we evaluate GDR on Inﬁniband fabric against a highly optimized Non-GDR baseline, that requires one extra copy but also uses RDMA transport over lossless Inﬁniband fabric 3 .”
Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems (2020, methodology): “The send operation is executed asynchronously through NVLink—we can send data to other GPUs simultaneously.”
Datacenter-Scale Analysis and Optimization of GPU Machine Learning Workloads (2021, methodology): “Tesla P100 or V100 GPU accelerators are connected by NVIDIA NVLink [10] to form an eight-GPU hybrid cube mesh.”