Understanding the Future of Energy Efficiency in Multi-Module GPUs

By Akhil Arunkumar, Evgeny Bolotin, David W. Nellans, Carole-Jean Wu. 2019

43 citations recorded by the source corpus.

—As Moore’s law slows down, GPUs must pivot towards multi-module designs to continue scaling performance at historical rates. Prior work on multi-module GPUs has focused on performance, while largely ignoring the issue of energy efﬁciency. In this work, we propose a new metric for GPU efﬁciency called EDP Scaling Efﬁciency that quantiﬁes the effects of both strong performance scaling and overall energy efﬁciency in these designs. To enable this analysis, we develop a novel top-down GPU energy estimation framework that is accurate within 10% of a recent GPU design. Being decoupled from granular GPU microarchitectural details, the framework is appropriate for energy efﬁciency studies in future GPUs. Using this model in conjunction with performance simulation, we show that the dominating factor inﬂuencing the energy efﬁciency of GPUs over the next decade is GPU-module (GPM) idle time. Furthermore, neither inter-module interconnect energy, nor GPM microarchitectural design is expected to play a key role in this regard. We demonstrate that multi-module GPUs are on a trajectory to become 2 × less energy efﬁcient than current monolithic designs; a signiﬁcant issue for data centers which are already energy constrained. Finally, we show that architects must be willing to spend more (not less) energy to enable higher bandwidth inter-GPM connections, because counter-intuitively, this additional energy expenditure can reduce total GPU energy consumption by as much as 45%, providing a path to energy efﬁcient strong scaling in the future.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering, Hardware and Architecture, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

In Hardware and Architecture

In Optimization & Training

Citations in the corpus

Chasing Carbon: The Elusive Environmental Footprint of Computing (2020, methodology): “Although optimizing nonfunctional properties, such as minimizing energy consumption through instruction-level energy annotation [98]–[100], can indirectly mitigate environmental impact, future compiler and programming-language technologies can optimize for CO2 emissions directly by applying our emis”
ACT: designing sustainable computer systems with an architectural carbon modeling tool (2022, background): “Unfortunately, unlike the myriad available tools to quantify hardware energy consumption [13, 20, 25, 47, 51, 53], there is a distinct lack of methodologies to understand the carbon footprint of system hardware.”