Anatomy of high-performance matrix multiplication

By Kazushige Goto, R. V. D. Geijn. TOMS, 2008

844 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/334f03c98b61090a9e956d4bfba4abdfa1c94b53

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

In Hardware and Architecture

Citations in the corpus

FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference (2021, methodology): “A key ingredient of fbgemm is performant low-precision gemm, which we have implemented using an approach similar to the one taken by other research works [3, 9, 11] targeting FP32 and FP64 data types but not low-precision.”
MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, background): “The inference operation has a large amount of computation; therefore, to efficiently handle such a large amount of computes with high throughput, they need to keep their necessary data in a shared cache as much as possible [27].”
Model-driven Level 3 BLAS Performance Optimization on Loongson 3A Processor (2012, methodology): “The sizes of submatrix˜A , ˜ B and C i are Mc × Kc, Kc × Nc and Mc × Nc. Goto's paper [3] observed that loading data from L2 cache only has trivial effects on floating-point performance on x86 CPUs.”
Large-Scale Learning of Embeddings with Reconstruction Sampling (2011, methodology): “NON-ZERO-INDICES(u) returns the set of non-zero indices in the row vector u. AXPY(α, A, B) = αA+B is part of the BLAS programming interface (Lawson et al., 1979).”
AUGEM: Automatically generate high performance Dense Linear Algebra kernels on x86 CPUs (2013, methodology): “For example, GotoBLAS, a highly opti mized library developed by Kazushige Goto [12], includes a large number of manually written assembly code varia tions to accommodate the numerous different hardware plat forms.”
Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer (2017, background): “Viewed this way, the trade-off we identify between the bandwidth of weights and input data becomes equivalent to the trade-off found when blocking the MW and MI matrices [16].”
BLIS: A Framework for Rapidly Instantiating BLAS Functionality (2015, neutral): “We now walk through the methods used to implement this functionality. operations, and thus the net impact on performance is, under modest assumptions, negligible Thus,the only code that must be optimized for a given architecture is that of the micro-kernel.”
Pieces of Eight: 8-bit Neural Machine Translation (2018, background): “Multilayer matrix multiply algorithms result in signiﬁcantly faster performance than naive algorithms (Goto and Geijn, 2008).”
Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures (2018, methodology): “Our attack can be generalized to other BLAS libraries, since all of them use blocked matrix-multiplication, and most of them implement Goto’s algorithm [41].”
Effortless Data Exploration with zenvisage: An Expressive and Interactive Visual Analytics System (2016, methodology): “To address this problem, we adapt techniques developed in high-performance computing— specifically, cache-based optimizations similar to those used in matrix multiplication [13].”