Accurate Post Training Quantization With Small Calibration Sets

By Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry. 2021

250 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (2022, positive): “Formally, this can be restated as Further, similar to (Nagel et al., 2020; Li et al., 2021; Frantar et al., 2022), we assume that the quantization grid for (cid:99) W is ﬁxed before the process, and that individual weights can move freely as in (Hubara et al., 2021; Frantar et al., 2022).”
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (2023, unknown): “Most PTQ methods, such as AdaRound [NAVB + 20], BitSplit [WCHC20], AdaQuant [HNH + 21], BRECQ [LGT + 21], or OBQ [FSA22] were designed for vision models or small-scale language models, with less than 100M parameters.”
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models (2022, background): “Among them, quantization [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], working in the low-precision arithmetic fashion, is one of the key approaches to compress large models and fit them into the lightweight devices.”
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2023, background): “Accurately quantizing models to extremely low precision (e.g., lower than 3 bits per parameter) typically requires more sophisticated data-dependent methods (Nagel et al., 2020; Wang et al., 2020; Hubara et al., 2021).”
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot (2023, background): “Post-training compression has traditionally been investigated in the context of quantization [23, 34, 29] to reduce the computational cost of quantization-aware training.”
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (2022, unknown): “Next, similarly to [19], we compute the layer-wise calibration losses (without augmentations) for all compression levels, corresponding to the models with exactly one layer compressed to a certain level.”
QuIP: 2-Bit Quantization of Large Language Models With Guarantees (2023, methodology): “There are many quantization methods which quantize by optimizing this proxy objective [5, 6, 9, 12, 14, 20, 32].”