An Entropy-based Pruning Method for CNN Compression
By Jian-Hao Luo, Jianxin Wu. 2017
206 citations recorded by the source corpus.
This paper aims to simultaneously accelerate and compress off-the-shelf CNN models via filter pruning strategy. The importance of each filter is evaluated by the proposed entropy-based method first. Then several unimportant filters are discarded to get a smaller CNN model. Finally, fine-tuning is adopted to recover its generalization ability which is damaged during filter pruning. Our method can reduce the size of intermediate activations, which would dominate most memory footprint during model training stage but is less concerned in previous compression methods. Experiments on the ILSVRC-12 benchmark demonstrate the effectiveness of our method. Compared with previous filter importance evaluation criteria, our entropy-based method obtains better performance. We achieve 3.3x speed-up and 16.64x compression on VGG-16, 1.54x acceleration and 1.47x compression on ResNet-50, both with about 1% top-5 accuracy decrease.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Taeyeong Kwak
- Inyoung Paik
- Balaraman Ravindran
- Yang He
- Sangjun Oh
- Bing Li
- Lingao Xiao
- Mitesh M. Khapra
In Computer Vision and Pattern Recognition
Citations in the corpus
- Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control (2019, methodology): “Although there are other methods in network pruning that claim to have better performance(Yu et al., 2018; Luo and Wu, 2017; Luo et al., 2017), we chose the Taylor criterion due to its computational efficiency.”
- Knowledge Distillation via Instance Relationship Graph (2019, methodology): “Specifically, network pruning [14, 15, 8, 17, 22] aims to delete unimportant connections of the trained network, while model quantization methods [5, 18, 3, 23] represent the float weights with fewer bits.”
- Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks (2018, methodology): “Entropy [29] : If the feature map corresponding to a filter produces the same output for every input (image) then this feature map and hence the corresponding filters may not be very important (because it does not play any discriminatory role).”
- Structured Pruning for Deep Convolutional Neural Networks: A Survey (2023, methodology): “Besides, the pruning process can be guided by leveraging the interpretations of a model [183], the loss landscape [110], generalization-stability trade-off [184] and entropy of a model [185].”