Benchmarking GPUs to tune dense linear algebra

By V. Volkov, J. Demmel. 2008

970 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Billion-Scale Similarity Search with GPUs (2017, background): “Making heavy use of register-resident data at the expense of occupancy or instead of shared memory is often proﬁtable [43].”
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters (2015, positive): “ of techniques have been developed to accelerate convolution on GPUs. Unlike CPUs, NVIDIA GPUs have an inverted memory hierarchy, where the register ﬁle is larger than the L1 cache. Volkov and Demmel [44] pioneered a communication-avoiding strategy to accelerate matrix multiplication on GPUs by stag”
Efficient Computation of k-Nearest Neighbour Graphs for Large High-Dimensional Data Sets on GPU Clusters (2013, methodology): “in [16] proposed an algorithm for a k-NN search by devising two kernels for distance calculation based on cuBLAS, [34] an optimized matrix multiplication library on the GPU, and a comparison kernel based on a parallel insertion sort, and achieved a 100-fold increase in speed compared with the approx”
State-of-the-art in heterogeneous computing (2010, methodology): “CUBLAS 2.0 uses the matrix multiplication al-gorithm developed by Volkov and Demmel [180], optimized not only to hide memory latency between main”
Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing (2012, methodology): “However, unlike the implementation released by Volkov, the Parboil GPU kernel code has been re-implemented such that the readability, configurability, and performance of the code are better balanced.”
A GPGPU compiler for memory optimization and parallelism management (2010, background): “Matrix multiplication (mm) is a commonly used algorithm and there has been continuing effort to improve its performance [18].”
From CUDA to OpenCL: Towards a performance-portable solution for multi-platform GPU programming (2012, methodology): “This work expanded on the work of Volkov and Demmel [14], who provided a high performance matrix multiply for NVIDIA’s previous generation GPUs.”
Towards dense linear algebra for hybrid GPU accelerated manycore systems (2009, methodology): “2), which led to the development of highly efficient factorizations [11,29,34] where a single CPU core is used to control the GPU and to factor the panels.”
The Scalable Heterogeneous Computing (SHOC) benchmark suite (2010, methodology): “Volkov and Demmel have also benchmarked NVIDIA GPUs using CUDA [13], with a primary focus on performance for linear algebra.”
Implementing sparse matrix-vector multiplication on throughput-oriented processors (2009, background): “Recently, Volkov and Demmel [21] and Barrachina et al. [2] have demon-c (cid:13) ACM, 2009.”
Inter-block GPU communication via fast barrier synchronization (2010, methodology): “As described in [29], the barrier function cannot guarantee that inter-block communication is correct unless a memory consistency model is assumed.”
Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU (2010, result): “Other studies which perform careful optimizations to CPU and GPU such as [27, 39, 40, 43, 49] report much lower speedup similar to ours.”
ArrayFire: a GPU acceleration platform (2012, neutral): “array left = get_image(left), right = get_image(right); array kernel= ones(5,5), sad = zeros(5,5); gfor (i = 0; i < nshifts; ++i) { array shifted = shift(right, 0, i); array diff = abs(left- shifted); sad(span,span,i) = convolve(diff, kernel); } array disparity = min(sad, 3);”
GPU-Accelerated Molecular Modeling Coming Of Age (2010, methodology): “An application program on the host uses attached GPUs through language extensions such as “Jacket” for Matlab 1 or PyCUDA 2 for Python, by calling libraries containing GPU-accelerated BLAS or FFT subroutines [3,37], or by invoking custom-writtenCUDAorOpenCLkernels.”
kNN Query Processing in Metric Spaces Using GPUs (2011, background): “On GPUs we are able to exploit fine-grained parallelism and it can be more efficient to just resort to brute-force algorithms, especially for highdimensional metric-spaces.”
Deep Rotation Equivariant Network (2017, background): “The theoretical analysis of memory usage is based on the fact that most of popular deep learning frameworks use GEMM algorithm[26] to compute image convolution.”