Reborn Filters: Pruning Convolutional Neural Networks with Limited Data

By Yehui Tang, Shan You, Jin Han, Chen Qian, Boxin Shi, Chao Xu, Changshui Zhang. 2020

33 citations recorded by the source corpus.

Channel pruning is effective in compressing the pretrained CNNs for their deployment on low-end edge devices. Most existing methods independently prune some of the original channels and need the complete original dataset to fix the performance drop after pruning. However, due to commercial protection or data privacy, users may only have access to a tiny portion of training examples, which could be insufficient for the performance recovery. In this paper, for pruning with limited data, we propose to use all original filters to directly develop new compact filters, named reborn filters, so that all useful structure priors in the original filters can be well preserved into the pruned networks, alleviating the performance drop accordingly. During training, reborn filters can be easily implemented via 1×1 convolutional layers and then be fused in the inference stage for acceleration. Based on reborn filters, the proposed channel pruning algorithm shows its effectiveness and superiority on extensive experiments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fei Wang
2. Yunhe Wang
3. Chunjing Xu
4. Kai Han
5. Chen Qian
6. Wei Zhang
7. Jianlong Wu
8. Xiaojie Li

In Computer Vision and Pattern Recognition

1. Yunhe Wang
2. Kai Han
3. Chunjing Xu
4. Fei Wang
5. Chen Qian
6. Dacheng Tao
7. Jianlong Wu
8. Tong Zhang

Citations in the corpus

• GreedyNAS: Towards Fast One-Shot NAS With Greedy Supernet (2020, background): “By dint of automatic feature engineering, deep neural networks (DNNs) have achieved remarkable success in various computer vision tasks, such as image classification [37, 36, 41, 33, 32, 15, 39], visual generation [34, 35], image retrieval [40, 7, 8, 12] and semantic comprehension [18, 17].”
• Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets (2020, methodology): “Pruning [11, 23, 39, 25, 45, 47, 24] is used to reduce the redundant parameters in neural networks that are insensitive to the model performance.”
• Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space (2020, background): “With this aim, efforts have been made to design more compact networks, compress or prune a pretrained model [28, 29] or even search for a tiny backbone network [37, 39].”
• Structured Pruning for Deep Convolutional Neural Networks: A Survey (2023, background): “Structured pruning [201] has lots of topics to investigate along with this new trend.”