Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning

By Tien-Ju Yang, Yu-hsin Chen, Vivienne Sze. 2016

797 citations recorded by the source corpus.

Deep convolutional neural networks (CNNs) are indispensable to state-of-the-art computer vision algorithms. However, they are still rarely deployed on battery-powered mobile devices, such as smartphones and wearable gadgets, where vision algorithms can enable many revolutionary real-world applications. The key limiting factor is the high energy consumption of CNN processing due to its high computational complexity. While there are many previous efforts that try to reduce the CNN model size or the amount of computation, we find that they do not necessarily result in lower energy consumption. Therefore, these targets do not serve as a good metric for energy cost estimation. To close the gap between CNN design and energy consumption optimization, we propose an energy-aware pruning algorithm for CNNs that directly uses the energy consumption of a CNN to guide the pruning process. The energy estimation methodology uses parameters extrapolated from actual hardware measurements. The proposed layer-by-layer pruning algorithm also prunes more aggressively than previously proposed pruning methods by minimizing the error in the output feature maps instead of the filter weights. For each layer, the weights are first pruned and then locally fine-tuned with a closed-form least-square solution to quickly restore the accuracy. After all layers are pruned, the entire network is globally fine-tuned using back-propagation. With the proposed pruning method, the energy consumption of AlexNet and GoogLeNet is reduced by 3.7x and 1.6x, respectively, with less than 1% top-5 accuracy loss. We also show that reducing the number of target classes in AlexNet greatly decreases the number of weights, but has a limited impact on energy consumption.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “Optimizing the overall energy efﬁciency of FL and on-device AI is an important ﬁrst step [92], [93], [94], [95], [96].”
What is the State of Neural Network Pruning? (2020, methodology): “These requirements not only increase infrastructure costs, but also make deployment of networks to resource-constrained environments such as mobile phones or smart devices challenging (Han et al., 2015; Sze et al., 2017; Yang et al., 2017).”
Wireless Network Intelligence at the Edge (2018, methodology): “Alternatively, one can optimize the pruning process based on energy consumption [201], as shown in Fig.”
Similarity-Preserving Knowledge Distillation (2019, background): “For example, unstructured pruning requires optimized sparse matrix multiplication routines to realize practical acceleration [26], platform constraint-aware compression [2, 36, 37] requires hardware simulators or empirical measurements, and arbitrarybit quantization [9, 17] requires specialized hard”
Bayesian Compression for Deep Learning (2017, methodology): “Inspired by this observation, several authors proposed pruning schemes that take these considerations into account [Wen et al., 2016, Yang et al., 2017] or even go as far as efﬁciency aware architectures to begin with [Iandola et al., 2017, Dong et al., 2017, Howard et al., 2017].”
Channel Pruning for Accelerating Very Deep Neural Networks (2017, neutral): “[52] prunes connections based on weights magnitude.”
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications (2018, methodology): “and this strategy allows reducing both filters and feature maps, which play an important role in resource consumption [25].”
SqueezeNext: Hardware-Aware Neural Network Design (2018, methodology): “To reduce the execution time and the energy consumption, the accelerator is designed to exploit the sparsity of the ﬁlter weights [7, 27] in the OS mode as well.”
I-BERT: Integer-only BERT Quantization (2021, neutral): “These techniques can be broadly categorized into (1) model pruning [18, 31, 35, 38, 40, 67], (2) knowledge distillation [21, 39, 43, 49, 70], (3) efficient neural architecture design [23, 24, 37, 51, 57], (4) hardware and neural architecture co-design [16, 17, 22, 29, 64], and (5) quantization [5, 7”
Designing Adaptive Neural Networks for Energy-Constrained Image Classification (2018, background): “First, prior art has investigated pruning-based methods to reduce the network connections [6, 11, 33].”
NeuralScale: Efficient Scaling of Neurons for Resource-Constrained Deep Neural Networks (2020, neutral): “Although this approach is widely used in other works [9, 29, 19, 51], it is also shown on several works [35, 20, 27, 16] that magnitude pruning does not result in an architecture with the best performance.”
AOWS: Adaptive and Optimal Network Width Search With Latency Constraints (2020, background): “M ( N ) can represent latency, power consumption constraints, or combinations of these objectives [29, 30].”
Structured Pruning for Deep Convolutional Neural Networks: A Survey (2023, background): “4 Pruning Targets Hardware [220], Energy [221], Robustness [222], [223], [224]”
Estimation of energy consumption in machine learning (2019, methodology): “body of research has emerged to optimize the energy-efficiency of these machine learning models and are driven by early energy modeling approaches applied in machine learning field [79].”
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers (2019, result): “In contrast to this work, the majority of preceding research on compute/memory efficient CNN inference has targeted CPUs and GPUs [33, 20, 63, 64, 50, 58, 53].”
Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices (2018, methodology): “Various different techniques have been widely explored in the de-sign of DNN models including reduced precision of weights and activations [3–8], compact network architectures [9–11] (i.e., compact DNNs), and increasing sparsity in the ﬁlter weights [12–14] (i.e., sparse DNNs).”