Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error

By Chunhui Jiang, Guiying Li, Chao Qian, Ke Tang. 2018

47 citations recorded by the source corpus.

Deep neural networks (DNNs) have achieved great success, but the applications to mobile devices are limited due to their huge model size and low inference speed. Much effort thus has been devoted to pruning DNNs. Layer-wise neuron pruning methods have shown their effectiveness, which minimize the reconstruction error of linear response with a limited number of neurons in each single layer pruning. In this paper, we propose a new layer-wise neuron pruning approach by minimizing the reconstruction error of nonlinear units, which might be more reasonable since the error before and after activation can change significantly. An iterative optimization procedure combining greedy selection with gradient decent is proposed for single layer pruning. Experimental results on benchmark DNN models show the superiority of the proposed approach. Particularly, for VGGNet, the proposed approach can compress its disk space by 13.6× and bring a speedup of 3.7×; for AlexNet, it can achieve a compression rate of 4.1× and a speedup of 2.2×, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mingkui Tan
2. Yong Guo
3. Junzhou Huang
4. Ting-Wu Chin
5. Diana Marculescu
6. Cha Zhang
7. Jin-Hui Zhu
8. Bohan Zhuang

In Computer Vision and Pattern Recognition

1. Mingkui Tan
2. Yong Guo
3. Junzhou Huang
4. Ting-Wu Chin
5. Diana Marculescu
6. Cha Zhang
7. Jin-Hui Zhu
8. Bohan Zhuang

Citations in the corpus

• Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks (2018, methodology): “…the solution for the problem (Gordon et al., 2018; Luo et al., 2017; He et al., 2017; Molchanov et al., 2017; He et al., 2018b; Yang et al., 2018a; Jiang et al., 2018; He et al., 2018a; Hu et al., 2016; Liu et al., 2017; Li et al., 2017; Lin et al., 2018; Wen et al., 2016) given its fine-grained…”
• Discrimination-Aware Network Pruning for Deep Model Compression (2020, methodology): “Moreover, for VGGNet, Adapt-DCP outperforms NRE [34] and DR [87] and obtains a 91.69% reduction in the model size and 69.81% in FLOPs.”