Towards Evolutionary Compression

By Yunhe Wang, Chang Xu, Jiayan Qiu, Chao Xu, Dacheng Tao. 2018

45 citations recorded by the source corpus.

Compressing convolutional neural networks (CNNs) is essential for transferring the success of CNNs to a wide variety of applications to mobile devices. In contrast to directly recognizing subtle weights or filters as redundant in a given CNN, this paper presents an evolutionary method to automatically eliminate redundant convolution filters. We represent each compressed network as a binary individual of specific fitness. Then, the population is upgraded at each evolutionary iteration using genetic operations. As a result, an extremely compact CNN is generated using the fittest individual, which has the original network structure and can be directly deployed in any off-the-shelf deep learning libraries. In this approach, either large or small convolution filters can be redundant, and filters in the compressed network are more distinct. In addition, since the number of filters in each convolutional layer is reduced, the number of filter channels and the size of feature maps are also decreased, naturally improving both the compression and speed-up ratios. Experiments on benchmark deep CNN models suggest the superiority of the proposed algorithm over the state-of-the-art compression methods, e.g. combined with the parameter refining approach, we can reduce the storage requirement and the floating-point multiplications of ResNet-50 by a factor of 14.64x and 5.19x, respectively, without affecting its accuracy.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Dacheng Tao
2. Chang Xu
3. Bochao Wang
4. Chao Xu
5. Hang Xu
6. Qi Tian
7. Zhaohui Yang
8. Chaoyue Wang

Citations in the corpus

• VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, methodology): “, 2020), QEA and EA-Pruning (Wang et al., 2018) on the CIFAR-10, CIFAR-100 and ImageNet datasets.”
• CARS: Continuous Evolution for Efficient Neural Architecture Search (2019, background): “EA-based approaches (Xie and Yuille 2017; Real et al. 2019; Wang et al. 2018; Shu et al. 2019) search architectures with the help of evolutionary algorithms.”
• Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts (2020, unknown): “Quantization [18, 19, 20, 21, 22, 23, 24] reduces the precision of the model weights, resulting in smaller model size and faster computation.”
• Evolutionary Generative Adversarial Networks (2018, methodology): “In addition, an evolutionary algorithm was proposed to compress deep learning models by automatically eliminating redundant convolution filters [61].”