Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

By Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, Yurong Chen. 2017

1,135 citations recorded by the source corpus.

This paper presents incremental network quantization (INQ), a novel method, targeting to efficiently convert any pre-trained full-precision convolutional neural network (CNN) model into a low-precision version whose weights are constrained to be either powers of two or zero. Unlike existing methods which are struggled in noticeable accuracy loss, our INQ has the potential to resolve this issue, as benefiting from two innovations. On one hand, we introduce three interdependent operations, namely weight partition, group-wise quantization and re-training. A well-proven measure is employed to divide the weights in each layer of a pre-trained CNN model into two disjoint groups. The weights in the first group are responsible to form a low-precision base, thus they are quantized by a variable-length encoding method. The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained. On the other hand, these three operations are repeated on the latest re-trained group in an iterative manner until all the weights are converted into low-precision ones, acting as an incremental network quantization and accuracy enhancement procedure. Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed method. Specifically, at 5-bit quantization, our models have improved accuracy than the 32-bit floating-point references. Taking ResNet-18 as an example, we further show that our quantized models with 4-bit, 3-bit and 2-bit ternary weights have improved or very similar accuracy against its 32-bit floating-point baseline. Besides, impressive results with the combination of network pruning and INQ are also reported. The code is available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations (2017, background): “It is worth noting that many recent works target quantization of the DNN model parameters and possibly the feature representation to speed up DNN evaluation on hardware with low-precision arithmetic, see, e.g. , [15, 23, 38, 43].”
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (2017, methodology): “Approaches that quantize only the weights ([2, 4, 8, 33]) are primarily concerned with on-device storage and less with computational efﬁciency.”
Searching for MobileNetV3 (2019, background): “Quantization [23, 25, 47, 41, 51, 52, 37] is another important complementary effort to improve the network efficiency through reduced precision arithmetic.”
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (2018, methodology): “Representative techniques include quantization [6, 34, 29, 54, 52, 21, 39, 53, 22, 3, 55, 9, 18], pruning [13, 12, 17, 36], low-rank decomposition [7, 8, 24, 27, 38, 51, 49], hashing [4], and deliberate architecture design [23, 19, 50].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Note that the calibration process is often conducted in parallel with the ﬁnetuning process for QAT. use a logarithmic distribution [175, 274], where the quantization steps and levels increase exponentially instead of linearly.”
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks (2018, background): “Some works [Zhu et al. , 2017; Zhou et al. , 2017] focus on compressing the CNNs by using low-precision weights .”
Training Quantized Nets: A Deeper Understanding (2017, background): “Previous work on obtaining a quantized neural network can be divided into two categories: quantizing pre-trained models with or without retraining [7, 13, 6, 14, 15], and training a quantized model from scratch [4, 5, 3, 1, 16].”
Discrimination-aware Channel Pruning for Deep Neural Networks (2018, background): “Recent studies on model compression mainly contain three categories, namely, quantization [31, 53], sparse or low-rank compressions [8, 9], and channel pruning [25, 26, 50, 48].”
Training and Inference with Integers in Deep Neural Networks (2018, methodology): “However, non-linear quantization method like logarithmic representation (Miyashita et al., 2016; Zhou et al., 2017)”
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks (2019, background): “Quantization [1, 5, 6, 7, 19, 28, 29, 30] is a very promising approach to address this problem by reducing the memory bottleneck, thus allowing the use of lower precision computational units in hardware.”
Fully Quantized Network for Object Detection (2019, background): “ory usage [33, 11, 30], quantizing both network parameters and activations to accelerate specific types of layers in the network using bitwise operations [35, 31, 14, 4], and quantizing network gradients to speed up distributed training [2].”
SPINN: synergistic progressive inference of neural networks over device and cloud (2020, methodology): “Our scheme differs from both i) weights-only reduction [17, 67, 86], which minimises the model size rather than activations’ size and ii) all-layers quantisation [14, 16, 24, 28] which requires complex techniques, such as quantisation-aware training [24, 28] or a re-training step [14, 16], to…”
Distilling Object Detectors with Task Adaptive Regularization (2020, background): “Weight quantization [29,32,9] refers to representing the weights with fewer bits, pruning approach [10,21,27] removes unimportant”
FINN-R (2018, methodology): “rmats, FINN-R must first perform a conversion to a common intermediate representation, in order to process the network. Currently, we support frontends for BinaryNet [13], Darknet [52] and Tensorpack [71]. In the case of BinaryNet or Tensorpack, FINN-R examines the dimensions of these frameworks’ ne”
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (2018, background): “Most previous works on accelerating CNNs can be roughly divided into four categories, namely, matrix decomposition [42, 35], low-precision weights [44, 43, 32], knowledge distilling [17, 19] and pruning.”
MetaQuant: Learning to Quantize by Learning to Penetrate Non-differentiable Quantization (2019, background): “The training process in Non-STE quantization suffers from heavy hyper-parameters tuning, such as weights partition portion in each step [15] and penalty setting in [10].”
A Lite Distributed Semantic Communication System for Internet of Things (2020, methodology): “Similarly, Zhou et al. [25] investigated an iterative quantization, which starts with a trained full-resolution model and then quantizes only a portion of the model followed by several epochs of re-training to recover the accuracy loss from quantization.”
Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm (2018, background): “[31] quantize the parameter incrementally and show that it is even possible to further reduce the weight precision to 2-5 bits with slightly higher accuracy than a fullprecision network on the ImageNet dataset.”