Quantized Convolutional Neural Networks for Mobile Devices

By Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, Jian Cheng. Computer Vision and Pattern Recognition, 2015

1,245 citations recorded by the source corpus.

Recently, convolutional neural networks (CNN) have demonstrated impressive performance in various computer vision tasks. However, high performance hardware is typically indispensable for the application of CNN models due to the high computation complexity, which prohibits their further extensions. In this paper, we propose an efficient framework, namely Quantized CNN, to simultaneously speed-up the computation and reduce the storage and memory overhead of CNN models. Both filter kernels in convolutional layers and weighting matrices in fully-connected layers are quantized, aiming at minimizing the estimation error of each layer's response. Extensive experiments on the ILSVRC-12 benchmark demonstrate 4 ~ 6× speed-up and 15 ~ 20× compression with merely one percentage loss of classification accuracy. With our quantized CNN model, even mobile devices can accurately classify images within one second.

Subfields: Computer Vision and Pattern Recognition, Efficiency & Compression

View Paper: https://arxiv.org/abs/1512.06473

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016, background): “For example, in [43], it is demonstrated that a convolutional neural network model can be sped up by a factor of 4-6 with minimal loss on classification accuracy on the ILSVRC-12 benchmark.”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, methodology): “There has been rising interest in building small and efﬁ-cient neural networks in the recent literature, e.g. [16, 34, 12, 36, 22].”
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices (2017, background): “Quantization [31, 27, 39, 45, 44] and factorization [22, 16, 18, 37] are proposed in literature to reduce redundancy in calculations to speed up inference.”
Paraphrasing Complex Network: Network Compression via Factor Transfer (2018, methodology): “Second, network quantization compresses a pretrained model by reducing the number of bits used to represent the weight parameters of the pretrained model (Rastegari et al., 2016; Wu et al., 2016).”
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “Quantization reduces the float-number representations of weights and activations to lower numerical precision (Wu et al., 2016; Han et al., 2015).”
Knowledge Distillation: A Survey (2020, methodology): “This category is further divided into model quantization (Wu et al. 2016), model binarization (Courbariaux et al. 2015), structural matrices (Sindhwani et al. 2015) and parameter sharing (Han et al. 2015; Wang et al. 2019f).”
Compressing Convolutional Neural Networks in the Frequency Domain (2015, background): “A few recent works [52, 61] also adopt the quantization techniques.”
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression (2017, neutral): “, the parameter quantization approach [33].”
Searching for MobileNetV3 (2019, background): “Quantization [23, 25, 47, 41, 51, 52, 37] is another important complementary effort to improve the network efficiency through reduced precision arithmetic.”
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients (2016, background): “Various approaches like quantization (Wu et al., 2015) and sparsiﬁcation (Han et al., 2015a) have also been proposed.”
Soft Weight-Sharing for Neural Network Compression (2017, background): “, 2015) or minimizes the estimation error (Wu et al., 2015).”
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (2018, background): “Network parameter reduction: Considerable efforts have been devoted to reducing the number of parameters in an existing network [13, 12, 17, 36, 7, 8, 24, 27, 38, 51, 49, 4, 35, 28, 45, 10, 46].”
Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning (2016, neutral): “2, the new pruning order is redeter-mined according to the new energy estimation of each layer.”
Adaptive Neural Networks for Efficient Inference (2017, background): “Reduced precision networks (Gong et al., 2014; Cour-bariaux et al., 2015; Chen et al., 2015; Hubara et al., 2016a; Wu et al., 2016; Rastegari et al., 2016; Hubara et al., 2016b) have been extensively studied to reduce the memory foot-print of networks and their test-time cost.”
More is Less: A More Complicated Network with Less Inference Complexity (2017, neutral): “Therefore, the formula of the LCCN can be written as: V ′ t (x, y) = k ′∑ i,j=1 C∑ c=1 W ′ t (i, j, c)U(x+ i− 1, y + i− 1, c) (2)”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “It is important to consider this fact for achieving optimal beneﬁts through hardware-aware quantization [86, 90, 238, 242, 246, 248, 257, 259].”
On the Efficacy of Knowledge Distillation (2019, background): “Some of this work has involved building more efficient neural network families [13, 7], pruning weights from larger neural networks [10], quantizing existing networks to use fewer bits for weights and activations [22] and distilling knowledge from larger networks into smaller ones [12, 1].”
I-BERT: Integer-only BERT Quantization (2021, methodology): “One promising method to tackle this challenge is quantization [15, 27, 28, 65, 66, 72], a procedure which compresses models into smaller size by representing parameters and/or activations with low bit precision, e.”