Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation

By Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P. Xing, Zhiqiang Shen. 2021

167 citations recorded by the source corpus.

The nonuniform quantization strategy for compressing neural networks usually achieves better performance than its counterpart, i.e., uniform strategy, due to its superior representational capacity. However, many nonuniform quantization methods overlook the complicated projection process in implementing the nonuniformly quantized weights/activations, which incurs non-negligible time and space overhead in hardware deployment. In this study, we propose Nonuniform-to-Uniform Quantization (N2UQ), a method that can maintain the strong representation ability of nonuniform methods while being hardware-friendly and efficient as the uniform quantization for model inference. We achieve this through learning the flexible inequidistant input thresholds to better fit the underlying distribution while quantizing these real-valued inputs into equidistant output levels. To train the quantized network with learnable input thresholds, we introduce a generalized straight-through estimator (G-STE) for intractable backward derivative calculation w.r.t. threshold parameters. Additionally, we consider entropy preserving regularization to further reduce information loss in weight quantization. Even under this adverse constraint of imposing uniformly quantized weights and activations, our N2UQ outperforms state-of-the-art nonuniform quantization methods by 0.5 ~ 1.7% on ImageNet, demonstrating the contribution of N2UQ design. Code and models are available at: https://github.com/liuzechun/Nonuniform-to-Uniform-Quantization.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Qiao
2. Ping Luo
3. Peng Gao
4. Wenqi Shao
5. Jianfei Cai
6. Jing Liu
7. Peng Xu
8. Bohan Zhuang

In Computer Vision and Pattern Recognition

1. Yu Qiao
2. Jianfei Cai
3. Dacheng Tao
4. Luc Van Gool
5. Ping Luo
6. Peng Gao
7. Xiaogang Wang
8. Chen Change Loy

Citations in the corpus

• Sharpness-aware Quantization for Deep Neural Networks (2021, methodology): “To encourage more information to be maintained by the quantized weights, several weight regularization methods [22, 44] have been proposed to alleviate the discrepancy between the full-precision and lowprecision weights.”
• OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models (2023, methodology): “…clipping (LWC) to reduce the difficulty of quantizing the weights in an LLM. Similar to previous methods with learnable clipping threshold (Esser et al., 2019; Liu et al., 2022; Choi et al., 2018), LWC also determines the optimal dynamic range of the weights by optimizing a clipping threshold.”
• LLM-QAT: Data-Free Quantization Aware Training for Large Language Models (2023, unknown): “For fine-tuning, it takes ~0.7 days ~0.8 days, and ~1.9 days to fine-tune the 7B-model and, 13B model and 30B model, respectively, with 8 A100 80G GPUs and batch size 1 per GPU on 100k generated examples.”