Structured Probabilistic Pruning for Deep Convolutional Neural Network Acceleration

By Huan Wang, Qiming Zhang, Yuehai Wang, Roland Hu. 2017

11 citations recorded by the source corpus.

Although deep Convolutional Neural Network (CNN) has shown better performance in various machine learning tasks, its application is accompanied by a significant increase in storage and computation. Among CNN simplification techniques, parameter pruning is a promising approach which aims at reducing the number of weights of various layers without intensively reducing the original accuracy. In this paper, we propose a novel progressive parameter pruning method, named Structured Probabilistic Pruning (SPP), which efficiently prunes weights of convolutional layers in a probabilistic manner. Unlike existing deterministic pruning approaches, in which the pruned weights of a well-trained model are permanently eliminated, SPP utilizes the relative importance of weights during training iterations, which makes the pruning procedure more accurate by leveraging the accumulated weight importance. Specifically, we introduce an effective weight competition mechanism to emphasize the important weights and gradually undermine the unimportant ones. Experiments indicate that our proposed method has obtained superior performance on ConvNet and AlexNet compared with existing pruning methods. Our pruned AlexNet achieves 4.0 $\sim$ 8.9x (averagely 5.8x) layer-wise speedup in convolutional layers with only 1.3\% top-5 error increase on the ImageNet-2012 validation dataset. We also prove the effectiveness of our method on transfer learning scenarios using AlexNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Song Han
2. Ji Lin
3. Zhijian Liu
4. Hanrui Wang
5. Chuang Gan
6. Yihui He
7. Li-Jia Li
8. Thomas S. Huang

In Computer Vision and Pattern Recognition

1. Song Han
2. Ji Lin
3. Zhijian Liu
4. Yi Yang
5. Luc Van Gool
6. Dacheng Tao
7. Chuang Gan
8. Trevor Darrell

Citations in the corpus

• AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “SPP [49] applies PCA analysis to each layer and takes the reconstruction error as the sensitivity measure to determine the pruning ratios.”