CNNpack: Packing Convolutional Neural Networks in the Frequency Domain

By Yunhe Wang, Chang Xu, Chao Xu, Dacheng Tao. 2016

197 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Kai Han
2. Yunhe Wang
3. Dacheng Tao
4. Jianyuan Guo
5. Chang Xu
6. Shan You
7. Yehui Tang
8. Hanting Chen

Citations in the corpus

• Data-Free Learning of Student Networks (2019, background): “[27] further introduced the discrete cosine transform (DCT) bases and converted convolution filters into the frequency domain to achieve higher compression and speed-up ratios.”
• Learning Instance-wise Sparsity for Accelerating Deep Models (2019, background): “In addition, [Wang et al., 2016] introduced the discrete cosine transform (DCT) bases and converted convolution filters into the frequency domain, thereby producing much higher compression ratio and speed improvement.”
• Co-Evolutionary Compression for Unpaired Image Translation (2019, methodology): “A straightforward method for reducing complexities of GANs can be directly borrowed from the conventional pruning methods [8, 32] for minimizing the reconstruction error on the output data, which can be formulated as a generatoraware loss function, i.”
• Frequency-Domain Dynamic Pruning for Convolutional Neural Networks (2018, background): “[15] decomposed the representations of convolutional filters in frequency domain as common parts (i.”
• Learning from Multiple Teacher Networks (2017, methodology): “CNNPack [32] treated filters as images, and adapted image compression techniques for help.”
• Post-Training Quantization for Vision Transformer (2021, methodology): “Among various compression methods like pruning [16, 29, 19] and weight decomposition [37], quantization method [9, 38, 7, 27, 14, 32] compresses a neural network by using lower bit-width for weight values without changing the model architecture, which is particularly useful for carefullydesigned net”
• Adversarial Learning of Portable Student Networks (2018, methodology): “To this end, there are a variety of methods have been proposed for compressing convolutional neural networks such as vector quantization (Gong et al. 2014), weight matrix decomposition (Denton et al. 2014), encoding (Chen et al. 2015), and pruning (Wang et al. 2016; Han, Mao, and Dally 2016).”
• Efficient Residual Dense Block Search for Image Super-Resolution (2019, background): “Recently, this automation has led to state of the art accuracy on classification problems (Zoph et al. 2018; Real et al. 2019; Wang et al. 2018b).”
• Reborn Filters: Pruning Convolutional Neural Networks with Limited Data (2020, methodology): “However, popular CNN models such as VGGNet (Simonyan and Zisserman 2014) and ResNet (He et al. 2016) have a huge number of model parameters and fail to meet the demands (Kim, Park, and Kwak 2018; Louizos, Ullrich, and Welling 2017; Wang et al. 2016; You et al. 2017b).”
• Towards Evolutionary Compression (2018, methodology): “Evaluation Original Trim [18] ThiNet [28] ECS DeepComp [13] CNNpack [44] ECS+CNNpack AlexNet [24] r The network accuracy was about 92 .”
• Learning Student Networks via Feature Embedding (2018, background): “[14] introduced the discrete cosine transform (DCT) bases and converted convolution filters into the frequency domain, thereby producing a much higher compression ratio and speed improvement.”
• AdderNet: Do We Really Need Multiplications in Deep Learning? (2019, unknown): “[31] further converted convolution filters into the DCT frequency domain and eliminated more floating number multiplications.”
• On Compressing Deep Models by Low Rank and Sparse Decomposition (2017, methodology): “[29] presented an effective CNN compression approach in the frequency domain using the discrete cosine transform and quantization strategies.”
• Focal Frequency Loss for Image Reconstruction and Synthesis (2020, background): “Besides, several studies have incorporated frequency analysis with network compression [16, 66, 68, 6, 17] and feature reduction [35, 61] to accelerate the training and inference of networks.”
• Frequency Domain Compact 3D Convolutional Neural Networks (2020, methodology): “[34] compressed filters in frequency domain with the help of discrete cosine transform.”