NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks

By Ermao Cai, Da-Cheng Juan, Dimitrios Stamoulis, Diana Marculescu. 2017

155 citations recorded by the source corpus.

"How much energy is consumed for an inference made by a convolutional neural network (CNN)?" With the increased popularity of CNNs deployed on the wide-spectrum of platforms (from mobile devices to workstations), the answer to this question has drawn significant attention. From lengthening battery life of mobile devices to reducing the energy bill of a datacenter, it is important to understand the energy efficiency of CNNs during serving for making an inference, before actually training the model. In this work, we propose NeuralPower: a layer-wise predictive framework based on sparse polynomial regression, for predicting the serving energy consumption of a CNN deployed on any GPU platform. Given the architecture of a CNN, NeuralPower provides an accurate prediction and breakdown for power and runtime across all layers in the whole network, helping machine learners quickly identify the power, runtime, or energy bottlenecks. We also propose the "energy-precision ratio" (EPR) metric to guide machine learners in selecting an energy-efficient CNN architecture that better trades off the energy consumption and prediction accuracy. The experimental results show that the prediction accuracy of the proposed NeuralPower outperforms the best published model to date, yielding an improvement in accuracy of up to 68.5%. We also assess the accuracy of predictions at the network level, by predicting the runtime, power, and energy of state-of-the-art CNN architectures, achieving an average accuracy of 88.24% in runtime, 88.34% in power, and 97.21% in energy. We comprehensively corroborate the effectiveness of NeuralPower as a powerful framework for machine learners by testing it on different GPU platforms and Deep Learning software tools.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

In ML Systems & Hardware

Citations in the corpus

Quantifying the Carbon Emissions of Machine Learning (2019, background): “This is something that should be taken into account by ML practitioners in their products that are deployed in real-world settings, for instance by using energy-efficient architectures [24] and computing infrastructure.”
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours (2019, background): “Prior art has showed that the total network latency of a mobile ConvNet can be modeled as the sum of each i-th layer’s runtime R, since the runtime of each operator is independent of other operators [3,4,21]:”
ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation (2018, background): “Thus, it is important to adjust and fine-tune the NN model to fit the energy constraint before deployment [5].”
Designing Adaptive Neural Networks for Energy-Constrained Image Classification (2018, methodology): “In addition, we plan to explore the use of predictive models that span different types of deep learning frameworks and commercial GPUs [3], and which could be flexibly extended to account for process variations [5, 27, 29] and thermal effects [2].”
AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning (2020, methodology): “For each type, we use methods that are widely adopted by existing works in this domain [10], [25], [42], [53], [121].”
Estimation of energy consumption in machine learning (2019, background): “NeuralPower [12] applies a regression based approach to model the power consumption and the run-time for a desktop GPU.”
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization (2019, background): “Prior art has showed that the total network latency of a mobile ConvNet can be modeled as the sum of each i-th layer’s runtime R, since the runtime of each operator is independent of other operators [38], [17], [16]:”
HyperPower: Power- and memory-constrained hyper-parameter optimization for neural networks (2017, methodology): “While we experimented with nonlinear regression formulations which can be plugged-in to the models ( e.g. , see our recent work [10]), these linear functions provide sufﬁcient accuracy (as shown in Section 5).”
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance (2020, methodology): “to execute DNN inference at the edge [8, 22, 36, 44, 46, 55, 89, 90, 92, 98].”