Fixed Point Quantization of Deep Convolutional Networks

By Darryl Dexu Lin, Sachin S. Talathi, V. Sreekanth Annapureddy. 2015

864 citations recorded by the source corpus.

In recent years increasingly complex architectures for deep convolution networks (DCNs) have been proposed to boost the performance on image recognition tasks. However, the gains in performance have come at a cost of substantial increase in computation and model storage resources. Fixed point implementation of DCNs has the potential to alleviate some of these complexities and facilitate potential deployment on embedded hardware. In this paper, we propose a quantizer design for fixed point implementation of DCNs. We formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers. Our experiments show that in comparison to equal bitwidth settings, the fixed point DCNs with optimized bit width allocation offer > 20%reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark.

Subfields: Computer Vision and Pattern Recognition, Model Architectures, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

In Efficiency & Compression

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “This topic has received a lot of attention in the broader ML community [194, 120, 436, 270, 312], but these methods either do not have a straightforward mapping to federated learning, or make the training process more complex which makes it difficult to adopt.”
Bayesian Compression for Deep Learning (2017, background): “Later studies extent this set to optimal fixed point [44] and hashing quantization [10].”
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (2018, background): “, uniform or logarithmic quantization) [11, 31, 53, 22, 37, 52] or otherwise pre-computed quantizers fixed during network training [3].”
Making AI Forget You: Data Deletion in Machine Learning (2019, background): “Deep Neural Networks and Stochastic Gradient Descent A line of research has observed the robustness of neural network training robustness to quantization and pruning [84, 46, 40, 71, 25, 52].”
Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021, methodology): “While prior work has demonstrated the feasibility of integer-only inference for computer vision models (Lin et al., 2016; Jacob et al., 2018; Krishnamoorthi, 2018; Zhang et al., 2018; Choukroun et al., 2019; Dong et al., 2019; Esser et al., 2019; Nagel et al., 2019, 2020), there is relatively…”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Speciﬁcally, the library leverages ﬁxed-point quantization [112, 151, 259] with power-of-two scaling factors so that quantization and dequantization processes can be carried out efﬁciently with bit shifting operations.”
Training Quantized Nets: A Deeper Understanding (2017, background): “Previous work on obtaining a quantized neural network can be divided into two categories: quantizing pre-trained models with or without retraining [7, 13, 6, 14, 15], and training a quantized model from scratch [4, 5, 3, 1, 16].”
Adding New Tasks to a Single Network with Weight Trasformations using Binary Masks (2018, methodology): “of additional parameters. Due to the low overhead required by the task-specic parameters introduced by our method, our work is linked to recent works on binarization [6,17,56,40,29] and quantization [18,28] of network parameters. Opposite to these works, we do not compress the whole architecture or ”
Softmax Tempering for Training Neural Machine Translation Models (2020, methodology): “We refer readers to orthogonal methods for speeding up NMT, such as weight pruning (See et al., 2016), quantization (Lin et al., 2016), and binarization (Courbariaux et al., 2017; Oda et al., 2017).”
Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming (2020, methodology): “• Greedy-compression: the complementary greedy approach (Lin et al., 2016) to sort the layers by their number of parameters and increase the precision of the layers from the smallest to the largest layer until the compression budget is reached.”
Accelerating Convolutional Neural Networks via Activation Map Compression (2018, methodology): “While we do not retrain our models after quantization, it has been shown by the literature to improve model accuracy [5, 37].”
YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration (2016, background): “It was shown that state-of-the-art results can be achieved quantizing the weights and activations of each layer separately [33], while lowering precision down to 2-bit ( − 1, 0, + 1) and increasing the network size [20].”
Triton: an intermediate language and compiler for tiled neural network computations (2019, background): “…proposed Triton-C, a higher-level language in which we were able to concisely implement efficient kernels for novel neural network architectures for CNNs. Directions of future work includes support for tensor cores, implementation of quantized kernels [26] and integration into higher level DSLs.”
Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers (2019, background): “Early works on quantization of deep networks targeted 16 bits fixed-point implementations (Lin et al., 2016), which result in an almost lossless approximation of full-precision trained networks, or extreme binarized networks, which, despite the fascinating low-computational and memory requirements, ”
Low-bit Quantization of Neural Networks for Efficient Inference (2019, methodology): “In order to better reconstruct those layers, some works use mixed precision [25], where some layers use more bits for representation.”
Knowledge Distillation for Recurrent Neural Network Language Modeling with Trust Regularization (2019, methodology): “To reduce the deep neural network model size, model quantization [5], model weight pruning [6], low rank matrix factorization [7] and knowledge distillation [8] are usually applied.”