Designing Adaptive Neural Networks for Energy-Constrained Image Classification

By Dimitrios Stamoulis, Ting-Wu Chin, Anand P. Krishnan, Haocheng Fang, S. Sajja, Mitchell Bognar, Diana Marculescu. 2018

67 citations recorded by the source corpus.

As convolutional neural networks (CNNs) enable state-of-the-art computer vision applications, their high energy consumption has emerged as a key impediment to their deployment on embedded and mobile devices. Towards efficient image classification under hardware constraints, prior work has proposed adaptive CNNs, i.e., systems of networks with different accuracy and computation characteristics, where a selection scheme adaptively selects the network to be evaluated for each input image. While previous efforts have investigated different network selection schemes, we find that they do not necessarily result in energy savings when deployed on mobile systems. The key limitation of existing methods is that they learn only how data should be processed among the CNNs and not the network architectures, with each network being treated as a blackbox. To address this limitation, we pursue a more powerful design paradigm where the architecture settings of the CNNs are treated as hyper-parameters to be globally optimized. We cast the design of adaptive CNNs as a hyper-parameter optimization problem with respect to energy, accuracy, and communication constraints imposed by the mobile device. To efficiently solve this problem, we adapt Bayesian optimization to the properties of the design space, reaching near-optimal configurations in few tens of function evaluations. Our method reduces the energy consumed for image classification on a mobile device by up to 6×, compared to the best previously published work that uses CNNs as blackboxes. Finally, we evaluate two image classification practices, i.e., classifying all images locally versus over the cloud under energy and communication constraints.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “Optimizing the overall energy efﬁciency of FL and on-device AI is an important ﬁrst step [92], [93], [94], [95], [96].”
Why Should We Add Early Exits to Neural Networks? (2020, methodology): “General hyper-parameter optimization procedures [ 65 ] can also be used to ﬁne-tune the entire architecture, including the eventual placement of some early exits.”
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours (2019, background): “Hardware-efficient ConvNets: While complex ConvNet designs have unlocked unprecedented performance levels in computer vision tasks, the accuracy improvement has come at the cost of higher computational complexity, making the deployment of state-of-the-art ConvNets to mobile devices challenging [19].”
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization (2019, background): “dented performance levels in computer vision tasks, the accuracy improvement has come at the cost of higher computational complexity, making the deployment of state-of-the-art ConvNets to mobile devices challenging [19].”
MnnFast: A Fast and Scalable System Architecture for Memory-Augmented Neural Networks (2019, background): “However, this optimization is not easy; therefore, researchers propose systematic accelerator performance analysis and optimization meth-ods [4, 23, 46, 58, 59, 63, 67, 68].”
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “[106] cast the adaptive DNN model selection issue as a hyperparameter optimization problem by taking into account the accuracy and communication constraints imposed by the devices.”
FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference (2019, methodology): “To reduce the model size, previous work has also conducted neural architecture search with energy constraint [21, 23, 24, 28].”
A systematic review of Green AI (2023, background): “Hence, most publications within this topic focus on identifying alternative strategies that reduce the number of iterations required to tune hyper-parameters (Stamoulis et al., 2018a).”
Surprisal-Triggered Conditional Computation with Neural Networks (2020, background): “In early exiting, a classifier is attached to an intermediate layer of a network and trained to determine whether stopping at that layer will result in a misclassification; if a misclassification is not predicted, subsequent layers are not computed [13, 14, 15, 16, 17].”