UNIQ
By Chaim Baskin, Eli Schwartz, Evgenii Zheltonozhskii, Natan Liss, Raja Giryes, A. Bronstein, A. Mendelson. 2018
43 citations recorded by the source corpus.
We present a novel method for neural network quantization. Our method, named UNIQ, emulates a non-uniform k-quantile quantizer and adapts the model to perform well with quantized weights by injecting noise to the weights at training time. As a by-product of injecting noise to weights, we find that activations can also be quantized to as low as 8-bit with only a minor accuracy degradation. Our non-uniform quantization approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We further propose a novel complexity metric of number of bit operations performed (BOPs), and we show that this metric has a linear relation with logic utilization and power. We suggest evaluating the trade-off of accuracy vs. complexity (BOPs). The proposed method, when evaluated on ResNet18/34/50 and MobileNet on ImageNet, outperforms the prior state of the art both in the low-complexity regime and the high accuracy regime. We demonstrate the practical applicability of this approach, by implementing our non-uniformly quantized CNN on FPGA.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yossi Adi
- Gabriel Synnaeve
- Alexandre D'efossez
- Junjie Yan
- Fisher Yu
- Xianglong Liu
- Jiazhen Lin
- Ruihao Gong
In Computer Vision and Pattern Recognition
- Junjie Yan
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Yi Yang
- Ming-Hsuan Yang
- Trevor Darrell
- Chunhua Shen
Citations in the corpus
- Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “2018a) and UNIQ (Baskin et al. 2018b) methods, which also rely on additive noise, DIFFQ achieves significantly better accuracy for the same model size.”
- Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks (2019, background): “Besides, there are other non-uniform methods like [30, 13, 3, 35, 37, 38, 44] which may need delicate hardware to get acceleration.”