Towards Accurate Post-training Network Quantization via Bit-Split and Stitching

By Peisong Wang, Qiang Chen, Xiangyu He, Jian Cheng. 2020

159 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (2022, methodology): “Alternatively, post-training methods (Nagel et al., 2020; Wang et al., 2020; Hubara et al., 2020; Nahshan et al., 2021), which compress the model in one shot, without retraining, would be very appealing.”
Post-Training Quantization for Vision Transformer (2021, methodology): “Bit-Split proposes a bit splitting and stitching framework [28], while the Top-1 accuracy degradation are 5.”
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction (2021, methodology): “Recent works, like Bit-Split (Wang et al., 2020) and AdaQuant (Hubara et al., 2020), also take this layer-wise objective to improve the post-training quantization.”
QuIP: 2-Bit Quantization of Large Language Models With Guarantees (2023, background): “There are other quantization procedures which do not round based on the proxy objective of [20], or are not designed for the largest language models [10, 11, 13, 19, 27, 28].”
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (2022, unknown): “Further, most works [42, 31, 19] focus on compressing linear and convolutional layers, which can be unfolded into linear ones, as these are prevalent in practice, and use the squared loss to measure the approximation error.”
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2023, background): “Accurately quantizing models to extremely low precision (e.g., lower than 3 bits per parameter) typically requires more sophisticated data-dependent methods (Nagel et al., 2020; Wang et al., 2020; Hubara et al., 2021).”
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (2023, unknown): “PTQ methods are a popular approach for one-shot compression of models with various sizes, based on a limited amount of calibration data, using accurate solvers, usually focused on layer-or group-wise compression sub-problems.”
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better (2021, background): “[156] try to retain the post-training quantization but with new heuristics to allocate the precision bits in a learned fashion.”