1.1 Computing's energy problem (and what we can do about it)

By Mark A. Horowitz. 2014

1,836 citations recorded by the source corpus.

Subfields: Electrical and Electronic Engineering, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In ML Systems & Hardware

Citations in the corpus

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations (2016, background): “Horowitz (2014) provides rough numbers for the energy consumed by the computation (the given numbers are for 45nm technology), as summarized in Tables 5 and 6.”
Binarized Neural Networks (2016, background): “Horowitz (2014) provides rough numbers for the computations’ energy consumption (the given numbers are for 45nm technology) as summarized in Tables 2 and 3.”
The hardware lottery (2020, background): “…a “big-ger is better” race in the number of model parameters that has gripped the ﬁeld of machine learning (Amodei et al., 2018), spi-ralling energy costs (Horowitz, 2014; Strubell et al., 2019) and the dizzying requirements of deploying machine learning to edge devices (Warden & Situnayake, 2019).”
Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021, background): “Low-bit ﬁxed-point representations, such as INT8, further reduce the energy consumption since the ﬁxed-point operations are more efﬁcient than their ﬂoating-point counterparts (Horowitz, 2014).”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “[96].”
Movement Pruning: Adaptive Sparsity by Fine-Tuning (2020, methodology): “iﬁcantly less energy greedy: the majority of the energy consumption comes from fetching the model parameters from the long term storage of the mobile device to its volatile memory [Han et al., 2016a, Horowitz, 2014]. Magnitude pruning [Han et al., 2015, 2016b], which preserves weights with high abso”
A comprehensive survey on model compression and acceleration (2020, background): “Each 32-bit floating-point addition and multiplication operation requires 0.9 pJ and 3.7 pJ of energy respectively (Horowitz 2014).”
Spike-driven Transformer (2023, methodology): “Refer to previous works[91, 70, 92, 27], we assume the data for various operations are 32-bit ﬂoating point implementation in 45nm technology [93], in which E MAC = 4 .”
Compact and Computationally Efficient Representation of Deep Neural Networks (2018, methodology): “TABLE I: Energy values (in pJ) of different elementary operations for a 45nm CMOS process [43].”
Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks (2021, background): “…more energetically expensive to calculate MACs than ACs (for example, 31-times more expensive on a 45 nm complementary metal–oxide–semiconductor 25 ), the relative efficiency of SNNs is determined by the number of connections multiplied by activity sparsity and the spiking neuron model…”