SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks

By Julian Faraone, Nicholas J. Fraser, Michaela Blott, Philip Heng Wai Leong. 2018

142 citations recorded by the source corpus.

Inference for state-of-the-art deep neural networks is computationally expensive, making them difficult to deploy on constrained hardware environments. An efficient way to reduce this complexity is to quantize the weight parameters and/or activations during training by approximating their distributions with a limited entry codebook. For very low-precisions, such as binary or ternary networks with 1-8-bit activations, the information loss from quantization leads to significant accuracy degradation due to large gradient mismatches between the forward and backward functions. In this paper, we introduce a quantization method to reduce this loss by learning a symmetric codebook for particular weight subgroups. These subgroups are determined based on their locality in the weight matrix, such that the hardware simplicity of the low-precision representations is preserved. Empirically, we show that symmetric quantization can substantially improve accuracy for networks with extremely low-precision weights and activations. We also demonstrate that this representation imposes minimal or no hardware implications to more coarse-grained approaches. Source code is available at https://www.github.com/julianfaraone/SYQ.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Similarity-Preserving Knowledge Distillation (2019, background): “Quantized networks [8, 15, 17, 30, 43, 46], such as fixed-point, binary, ternary, and arbitrary-bit networks, encode weights and/or activations using a small number of bits, or at lower precision.”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “…each branch, and we refer the interested readers to [191] for more details. a) Quantization Error Minimization: The ﬁrst branch of solutions aims to minimize the quantization error, i.e., the gap between the real values and the quantized values [19, 34, 62, 102, 149, 155, 160, 165, 174, 211, 240].”
Learning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss (2018, unknown): “0 SYQ† [6] - - - - - - - - - - - 55.”
Training Compact Neural Networks with Binary Weights and Low Precision Activations (2018, methodology): “SYQ employs binary weights and 8-bit activations.”
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions (2020, unknown): “3 SYQ [10] 1/2 55.”
Relaxed Quantization for Discretized Neural Networks (2018, unknown): “In Figure 4a we observe that on ResNet-18 the RQ variants form the “Pareto frontier” in the trade-off between accuracy and efﬁciency, along with SYQ, Apprentice and Jacob et al. (2017).”
Towards Accurate Post-training Network Quantization via Bit-Split and Stitching (2020, background): “…et al., 2017b; Zhu et al., 2019; Li et al., 2017; Tang et al., 2017), ﬁxed-point factorization (Wang & Cheng, 2017; Hu et al., 2018a), learnable quantization (Faraone et al., 2018; Wang et al., 2020), etc. Post-training Quantization: Post-training quantization has recently drawn much attention.”
Low-bit Quantization of Neural Networks for Efficient Inference (2019, background): “For example, with the efﬁ-cient group-wise [1, 27], channel-wise or pixel-wise quantization settings [8], the dot product must be split into several dot products to be accumulated together.”
T-Basis: a Compact Representation for Neural Networks (2020, background): “Popular quantization schemes include lower precision (Wu et al., 2016; Jacob et al., 2018; Faraone et al., 2018; Nagel et al., 2019), binary (Courbariaux et al., 2015; 2016; Rastegari et al., 2016; Zhou et al., 2016b), or even ternary (Zhu et al., 2017) weights.”
Binary Neural Networks: A Survey (2020, methodology): “The work of Faraone et.al. groups parameters in training process and gradually quantizes each group with optimized scaling factor to minimize the quantization error [77].”