Latency lags bandwith

By David A. Patterson. 2004

280 citations recorded by the source corpus.

Subfields: Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

Citations in the corpus

Tiered-latency DRAM: A low latency and low cost DRAM architecture (2013, unknown): “DRAM Capacity & Latency Over Time [5, 19, 35, 43]”
Storage-class memory: The next storage system technology (2008, background): “Patterson [6] observes that latency improvements are lagging bandwidth improvements throughout the computing industry.”
Why flow-completion time is the right metric for congestion control (2006, background): “Plot inspired by Patterson’s illustration of how latency lags bandwidth in computer systems [2].”
AI and Memory Wall (2024, unknown): “The memory wall problem involves both the limited capacity, the bandwidth of memory transfer, as well as its latency (which has been even harder to improve [32] than bandwidth).”
A case for flash memory ssd in enterprise database applications (2008, background): “Nonetheless, for more than a decade in the past, the latency of disk has improved at a much slower pace than the bandwidth of disk, and the latency-bandwidth imbalance is expected to be even more evident in the future [19].”
The Landscape of Parallel Computing Research: A View from Berkeley (2006, unknown): “[Patterson 2004] 7.”
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding (2024, background): “However, autoregressive decoding is limited by its inference latency, which primarily stems from the memory-bound computation of LLMs (Patterson, 2004; Shazeer, 2019).”
Roofline: an insightful visual performance model for multicore architectures (2009, background): “THE ROOFLINE MODEL We believe that for the recent past and foreseeable future, off-chip memory bandwidth will often be the constraining resource[23].”
SqueezeLLM: Dense-and-Sparse Quantization (2023, background): “The disparity between compute and memory bandwidth, along with the growing memory requirements of deep learning, has been termed the Memory Wall problem [20].”
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty (2024, background): “Inference in LLMs is memory-bound (Patterson, 2004; Shazeer, 2019), leaving GPU computational resources un-derutilized.”