Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures

By Samuel Williams, Andrew Waterman, David A. Patterson. 2008

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

HellaSwag: Can a Machine Really Finish Your Sentence? (2019, methodology): “This estimation is a rough estimate: we convert reported TPU runtimes to our benchmark RTX 2080 Ti GPU using the Roofline model (Williams et al., 2009), which focuses primarily on the bottleneck of loading tensors into GPU memory.”
Beyond human-level accuracy: computational challenges in deep learning (2019, methodology): “The estimates use the Roofline model to predict the overall system throughput given the full-graph algorithmic FLOPs and memory accesses [38].”
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022, methodology): “This is commonly measured by the arithmetic intensity [82], which is the number of arithmetic operations per byte of memory access.”
Billion-Scale Similarity Search with GPUs (2017, methodology): “For any CPU or GPU algorithm, either memory or arithmetic throughput should be the limiting factor as per the roofline performance model [48].”
The case for 4-bit precision: k-bit Inference Scaling Laws (2022, methodology): “To test this claim we construct a roofline model (Williams et al., 2009) that is based on real matrix multiplication data as implemented by NVIDIA.”
Once for All: Train One Network and Specialize it for Efficient Deployment (2019, methodology): “Removing the memory bottleneck results in higher utilization and GOPS/s by 70%-90%, pushing the operation point to the upper-right in the rooﬂine model (Williams et al., 2009), as shown in Figure 13.”
HAQ: Hardware-Aware Automated Quantization With Mixed Precision (2018, methodology): “We explain the difference of quantization policy between edge and cloud by the roofline model [28].”
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications (2018, methodology): “We apply a roofline model for each layer, where each layer differs in whether it reads activations/weights from off- or on-chip memory based on a simple greedy on-chip memory allocation algorithm [72].”
FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference (2021, background): “The figure also shows the roofline [10] peak for the same machine.”
SqueezeLLM: Dense-and-Sparse Quantization (2023, unknown): “A typical metric used to assess whether a kernel is compute-bound or memory-bound is arithmetic intensity [71], which is the ratio of compute operations to memory operations in that kernel.”
ST-MoE: Designing Stable and Transferable Sparse Expert Models (2022, background): “The compute-to-memory ratio (operational intensity) can serve as an estimate of the efﬁciency of different operations (Williams et al., 2009; Shazeer, 2019).”
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference (2022, methodology): “Unlike RETRO, FiDO’s efficiency improvements allow it to tractably attend to many retrieved passages with a much larger decoder. ing factor for computations on modern devices (Williams et al., 2009; Dao et al., 2022; Shazeer, 2019).”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, methodology): “model is an intuitive visual performance model used to relate the attainable performance to the peak performance that can be provided by the hardware platform and the off-chip memory traffic [150].”
Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product (2021, background): “For 3 apps the CMEM gain is even higher at 1.7 –2.2X, providing much greater benefits. roofline model [46] in Figure 12 helps explain what is going on.”
Automating model search for large scale machine learning (2015, methodology): “In addition, we use a rooﬂine [47] inspired model of the tradeoff between job-speciﬁc memory and compute requirements.”
Benchmarking TPU, GPU, and CPU Platforms for Deep Learning (2019, methodology): “In this section, we use the roofline model [56] to analyze the computation and memory bandwidth of FCs and CNNs.”
Caffeine: Towards uniformed representation and acceleration for deep convolutional neural networks (2016, methodology): “The roofline model [31] is initially proposed in multicore systems to provide insight analysis of attainable performance by relating processors’ peak computation performance and the off-chip memory traffic.”