LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

By Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua. 2018

770 citations recorded by the source corpus.

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Efficiently Scaling Transformer Inference (2022, neutral): “…(Sanh et al., 2019; Sun et al., 2020), and model compression techniques, such as pruning (Li et al., 2020b; Brix et al., 2020; Zhou et al., 2021; Li et al., 2020a; Wang et al., 2020), or quantization (Dettmers et al., 2022; Abdolrashidi et al., 2021; Zafrir et al., 2019; Zhang et al., 2018).”
The case for 4-bit precision: k-bit Inference Scaling Laws (2022, background): “…centering (Krishnamoorthi, 2018; Jacob et al., 2017), learned data types that are found through clustering (Gong et al., 2014; Han et al., 2015; Choi et al., 2016; Park et al., 2017), or direct codebook optimization (Rastegari et al., 2016; Hou et al., 2016; Leng et al., 2018; Zhang et al., 2018).”
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “For example, using Hessian information (Dong et al., 2019), step-size quantization (Esser et al., 2019), soft quantization (Gong et al., 2019), mixed-precision via linear programming optimization (Yao et al., 2021), and other learned quantization methods (Zhang et al., 2018; Gholami et al., 2021).”
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT (2019, methodology): “Here we solely focus on quantization [4, 6, 8, 14, 17, 24, 40, 42].”
Similarity-Preserving Knowledge Distillation (2019, background): “Quantized networks [8, 15, 17, 30, 43, 46], such as fixed-point, binary, ternary, and arbitrary-bit networks, encode weights and/or activations using a small number of bits, or at lower precision.”
Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation (2021, background): “There are many effective strategies for improving the parameter efficiency of trained networks: for example, pruning algorithms learn to remove redundant or less important parameters [12,40], and quantization algorithms learn to represent parameters using fewer bits while maintaining task performanc”
Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “As data augmentation, we used color-jitter, random erase, and either cutmix or mixup (Zhang et al. 2018b).”
Single Path One-Shot Neural Architecture Search with Uniform Sampling (2019, background): “Evaluation is on ResNet-18 and ResNet-34 [7], as common practice in previous quantization works (e.g. [5, 27, 16, 36, 31]).”
Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021, methodology): “While prior work has demonstrated the feasibility of integeronly inference for computer vision models (Lin et al., 2016; Jacob et al., 2018; Krishnamoorthi, 2018; Zhang et al., 2018; Choukroun et al., 2019; Dong et al., 2019; Esser et al., 2019; Nagel et al., 2019, 2020), there is relatively little ”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, positive): “These methods are referred to as learnable quantizers, and the quantization steps/levels are generally trained with iterative optimization [250, 268] or gradient descent [123, 155, 256].”
BinaryBERT: Pushing the Limit of BERT Quantization (2020, background): “Existing binarization approaches can be categorized into quantization error minimization (Raste-gari et al., 2016; Hou et al., 2017; Zhang et al., 2018), improving training objectives (Martinez et al., 2020; Bai et al., 2020) and reduction of gradient mismatch (Bai et al., 2018; Liu et al., 2018,…”
I-BERT: Integer-only BERT Quantization (2021, methodology): “One promising method to tackle this challenge is quantization [15, 27, 28, 65, 66, 72], a procedure which compresses models into smaller size by representing parameters and/or activations with low bit precision, e.”
Learned Step Size Quantization (2019, methodology): “…al. (2016), later work focused on ﬁtting the quantizer to the data, either based on statistics of the data distribution (Li & Liu, 2016; Zhou et al., 2016; Cai et al., 2017; McKinstry et al., 2018) or seeking to minimize quantization error during training (Choi et al., 2018c; Zhang et al., 2018).”
QKD: Quantization-aware Knowledge Distillation (2019, methodology): “Our QKD is compared with various methods such as AP* [30], LQ-Net [50], PACT [6], QIL [18] and DSQ [10].”
ZeroQ: A Novel Zero Shot Quantization Framework (2020, methodology): “Here we focus on quantization [2, 6, 34, 41, 23, 48, 45, 46, 5, 8, 42], which compresses the model by reducing the bit precision used to represent parameters and/or activations.”
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks (2020, background): “To reduce the performance gap between the quantized model and its full precision counterpart, quantization-aware training ( QAT ) ﬁne-tunes the quantized model on the training dataset [6, 7, 8, 9, 10, 11, 12, 13, 14].”
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks (2020, positive): “…data distribution (Zhou et al. 2017), stochastic rounding (Gupta et al. 2015), weight cluster (Park, Ahn, and Yoo 2017), trainable quantization (Zhang et al. 2018), ﬁttable quantization scale (Cai et al. 2017), pow2-ternaization (Ott et al. 2016), stochastic weight averaging (Shin, Boo, and…”
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions (2020, methodology): “t binary network [3] by 4.0% top-1 accuracy while incurring only half the OPs1. 2 Related Work There have been extensive studies on neural network compression and acceleration, including quantization [46,39,43], pruning [9,12,24,22], knowledge distillation [14,33,6] and compact network design [15,32”