Cascaded Projection: End-To-End Network Compression and Acceleration

By Breton L. Minnehan, Andreas Savakis. 2019

27 citations recorded by the source corpus.

We propose a data-driven approach for deep convolutional neural network compression that achieves high accuracy with high throughput and low memory requirements. Current network compression methods either find a low-rank factorization of the features that requires more memory or select only a subset of features by pruning entire filter channels. We propose the Cascaded Projection (CaP) compression method that projects the output and input filter channels of successive layers to a unified low dimensional space based on a low-rank projection. We optimize the projection to minimize classification loss and the difference between the next layer's features in the compressed and uncompressed networks. To solve this non-convex optimization problem we propose a new optimization method of a proxy matrix using backpropagation and Stochastic Gradient Descent (SGD) with geometric constraints. Our cascaded projection approach leads to improvements in all critical areas of network compression: high accuracy, low memory consumption, low parameter count and high processing speed. The proposed CaP method demonstrates state of the art results compressing VGG16 and ResNet networks with over 4X reduction in the number of computations and excellent performance in top-5 accuracy on the ImageNet dataset before and after fine-tuning.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Xuanyi Dong
2. Yi Yang

In Computer Vision and Pattern Recognition

1. Xuanyi Dong
2. Yi Yang
3. Liang Zheng
4. Luc Van Gool
5. Xiaogang Wang
6. Wanli Ouyang
7. Dacheng Tao
8. Jitendra Malik

Citations in the corpus

• Network Pruning via Transformable Architecture Search (2019, background): “[34] transferred the knowledge of uncompressed network via a block-wise reconstruction loss.”
• Learning Filter Basis for Convolutional Neural Network Compression (2019, unknown): “to the same level. Model Method Top-1 Err.(%) / Baseline FLOPs (%) VGG K-means [46] 6.24 / 5.98 100 Factor [47] 7.12 / 5.98 36.6 Group [41] 6.69 / 5.98 46.1 Basis (ours) 6.23 / 5.98 23.5 ResNet56 CaP [38] 6.78 / 6.49 50.2 ENC [22] 7.00 / 6.90 50.0 AMC [17] 8.10 / 7.20 50.0 KSE [32] 6.77 / 6.97 48.0 ”