Post training 4-bit quantization of convolutional networks for rapid-deployment

By Ron Banner, Yury Nahshan, Daniel Soudry. Neural Information Processing Systems, 2018

768 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/f789425a7af1d012675118d7d10cd50afad09074

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021, methodology): “…2018b; Zhu et al., 2020); running min-max uses exponential moving average of the min and max over multiple batches (Krishnamoorthi, 2018); MSE ﬁnds quantization parameters that minimize mean squared error between quantized and ﬂoating-point tensors (Choukroun et al., 2019; Banner et al., 2018).”
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations (2023, unknown): “To relieve this issue, researchers have proposed various neural network compression techniques to design lightweight models, including neural network pruning ([17]), low-rank factorizations of the weight matrices ([18, 19]), quantization ([11, 20]), knowledge distillation ([21]), neural architecture”
HAWQV3: Dyadic Neural Network Quantization (2020, background): “Other work tries to include periodic regularization to assist quantization [17, 38], apply post training quantization [6, 7, 26], or improve accuracy by changing the channel counts accordingly for different layers [11].”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, methodology): “Post training quantization involves taking a trained model, quantizing the weights, and then re-optimizing the model to generate a quantizedmodel with scales [16].”
The Knowledge Within: Methods for Data-Free Model Compression (2019, background): “We argue that more advanced calibration methods [21, 1] may improve results as they are uncoupled to the proposed data generation schemes.”
BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning (2020, methodology): “These quantization methods are mainly used to compress values for transmission [58] or accelerate inference where only multiplication is needed [5].”
Towards Unified INT8 Training for Convolutional Neural Network (2019, background): “As a result, many recent studies have been focusing on how to accelerate the inference of neural networks by fixed-point quantization on weights or activations [6, 3, 24, 29, 27, 64, 42, 52, 63, 54, 21, 44, 57], and design dedicated hardware utilizing the efficient integer arithmetic [17, 5, 26, 23]”
Loss aware post-training quantization (2019, methodology): “Multiple alternative schemes for the correction of quantization bias were proposed, and those techniques are widely applied in state-of-the-art quantization approaches (Banner et al. 2018; Finkelstein et al. 2019; Nagel et al. 2019; Fang et al. 2020).”
Up or Down? Adaptive Rounding for Post-Training Quantization (2020, methodology): “On the other hand, much attention has recently been dedicated to post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice.”
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (2022, unknown): “Although our bias and variance correction step applied to YOLO and BERT models is similar to the schemes described in [32] and [1], we now describe our exact procedure for additional clarity: 1.”
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks (2020, background): “Among inference quantization approaches, post training quantization usually does not require access to the partial/full training dataset, and it does not need to re-train/ﬁne-tune the quantized model [1, 2, 3, 4, 5].”
Towards Accurate Post-training Network Quantization via Bit-Split and Stitching (2020, methodology): “The authors of (Nahshan et al., 2019) study post-training quantization from the viewpoint of loss landscape.”
F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization (2022, methodology): “…network compression (Courbariaux et al., 2015; Han et al., 2015; Zhu et al., 2016; Zhou et al., 2016; 2017; Mishra et al., 2017; Park et al., 2017; Banner et al., 2018), with two types of training strategies: Post-Training Quantization directly quantizes a pre-trained full-precision model (He &…”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, unknown): “2) Post-Training Quantization: An alternative to the expensive QAT method is Post-Training Quantization (PTQ) which performs the quantization and the adjustments of the weights, without any ﬁne-tuning [11, 24, 40, 60, 61, 68, 69, 88, 107, 140, 146, 170, 178, 273].”
Post-Training Quantization on Diffusion Models (2022, unknown): “In order to set quantization parameters for the weight tensor and the activation tensor in a layer, a simple but effective way is to select the quantization parameters that minimize the MSE of the tensors before and after quantization [1, 5, 42, 12].”
LSQ+: Improving low-bit quantization through learnable offsets and better initialization (2020, methodology): “Post-training quantization methods [2, 29, 6] optimize neural networks for quantization without full training and using a little amount of data.”
Post-Training Quantization for Vision Transformer (2021, methodology): “[2] present the so-called ACIQ method to analytically compute the clipping range, as well as the per-channel bit allocation for NNs.”
A White Paper on Neural Network Quantization (2021, methodology): “The optimization problem is commonly solved using grid search, golden section method or analytical approximations with closed-form solution (Banner et al., 2019).”