Summarizing CPU and GPU Design Trends with Product Data

By Yifan Sun, N. Agostini, Shi Dong, D. Kaeli. 2019

108 citations recorded by the source corpus.

Moore's Law and Dennard Scaling have guided the semiconductor industry for the past few decades. Recently, both laws have faced validity challenges as transistor sizes approach the practical limits of physics. We are interested in testing the validity of these laws and reflect on the reasons responsible. In this work, we collect data of more than 4000 publicly-available CPU and GPU products. We find that transistor scaling remains critical in keeping the laws valid. However, architectural solutions have become increasingly important and will play a larger role in the future. We observe that GPUs consistently deliver higher performance than CPUs. GPU performance continues to rise because of increases in GPU frequency, improvements in the thermal design power (TDP), and growth in die size. But we also see the ratio of GPU to CPU performance moving closer to parity, thanks to new SIMD extensions on CPUs and increased CPU core counts.

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Duen Horng Chau
2. Noam Shazeer
3. Orhan Firat
4. Z. Chen

Citations in the corpus

• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, background): “It is reported that GPU price per flops dropped a factor of ten in just the last 4 years [73] and flops per watts increased by 2 magnitude over the past 12 years [74].”
• EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models (2021, methodology): “To implement hardware alternatives, we utilize a dataset from Sun et al. [20], consisting of power draw 𝑝 and floating point operations per second (FLOPS) 𝑠 for over 4000 GPUs and CPUs.”