Jointly Sparse Convolutional Neural Networks in Dual Spatial-winograd Domains
By Yoojin Choi, Mostafa El-Khamy, Jungwon Lee. 2019
6 citations recorded by the source corpus.
We consider the optimization of deep convolutional neural networks (CNNs) such that they provide good performance while having reduced complexity if deployed on either conventional systems with spatial-domain convolution or lower-complexity systems designed for Winograd convolution. The proposed framework produces one compressed model whose convolutional filters can be made sparse either in the spatial domain or in the Winograd domain. Hence, the compressed model can be deployed universally on any platform, without need for re-training on the deployed platform. To get a better compression ratio, the sparse model is compressed in the spatial domain that has a fewer number of parameters. From our experiments, we obtain 24.2× and 47.7× compressed models for ResNet-18 and AlexNet trained on the ImageNet dataset, while their computational cost is also reduced by 4.5× and 5.1×, respectively.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
In Computer Vision and Pattern Recognition
- Jonathan Frankle
- John V. Guttag
- Davis W. Blalock
- Jose Javier Gonzalez Ortiz
- Michael Carbin
- Zhangyang Wang
- Michael Crawshaw
- Tianlong Chen
Citations in the corpus
- What is the State of Neural Network Pruning? (2020, methodology): “We found up to a factor of four variation in the reported FLOPs of different papers for the same architecture and dataset, with (Yang et al., 2017) reporting 371 MFLOPs for AlexNet on ImageNet, (Choi et al., 2019) reporting 724 MFLOPs, and (Han et al., 2015) reporting 1500 MFLOPs.”