Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking

By Haichuan Yang, Yuhao Zhu, Ji Liu. 2018

41 citations recorded by the source corpus.

Deep Neural Networks (DNNs) are increasingly deployed in highly energy-constrained environments such as autonomous drones and wearable devices while at the same time must operate in real-time. Therefore, reducing the energy consumption has become a major design consideration in DNN training. This paper proposes the first end-to-end DNN training framework that provides quantitative energy consumption guarantees via weighted sparse projection and input masking. The key idea is to formulate the DNN training as an optimization problem in which the energy budget imposes a previously unconsidered optimization constraint. We integrate the quantitative DNN energy estimation into the DNN training process to assist the constrained optimization. We prove that an approximate algorithm can be used to efficiently solve the optimization problem. Compared to the best prior energy-saving methods, our framework trains DNNs that provide higher accuracies under same or lower energy budgets. Code is publicly available.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks (2018, methodology): “For example, past work has used (cid:96) 2 of ﬁlter weights (He et al., 2018a; Yang et al., 2018b), (cid:96) 1 of ﬁlter weights (Mao et al., 2017; Li et al., 2017), (cid:96) 2 of ﬁlter weights and ﬁlter weights of the next layer (Jiang et al., 2018), ﬁrst-order Taylor expansion on the loss…”