Bayesian Bits: Unifying Quantization and Pruning

By Mart van Baalen, Christos Louizos, Markus Nagel, Rana Ali Amjad, Tijmen Blankevoort, Max Welling. 2020

143 citations recorded by the source corpus.

We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition of the quantization operation, which sequentially considers doubling the bit width. At each new bit width, the residual error between the full precision value and the previously rounded value is quantized. We then decide whether or not to add this quantized residual error for a higher effective bit width and lower quantization noise. By starting with a power-of-two bit width, this decomposition will always produce hardware-friendly configurations, and through an additional 0-bit option, serves as a unified view of pruning and quantization. Bayesian Bits then introduces learnable stochastic gates, which collectively control the bit width of the given tensor. As a result, we can obtain low bit solutions by performing approximate inference over the gates, with prior distributions that encourage most of them to be switched off. We experimentally validate our proposed method on several benchmark datasets and show that we can learn pruned, mixed precision networks that provide a better trade-off between accuracy and efficiency than their static bit width equivalents.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fast Lossless Neural Compression with Integer-Only Discrete Flows (2022, background): “…and activations in low precision format instead of 32-bit ﬂoating-point numbers (Courbariaux et al., 2015; Hubara et al., 2016; Rastegari et al., 2016; Zhou et al., 2016; Jacob et al., 2018; Choi et al., 2018; Dong et al., 2019; Esser et al., 2020; Chen et al., 2020a; Van Baalen et al., 2020).”
Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021, background): “Finally, it is possible to assign different bit-widths for different layers or parts of the network, a technique known as mixed precision (Lin et al., 2016; Wu et al., 2018a; Zhou et al., 2018; Dong et al., 2019; Wang et al., 2019; van Baalen et al., 2020).”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Recent work has shown that low precision and mixed-precision quantization with INT4/INT8 works in practice [51, 81, 101, 107, 182, 193, 204, 231, 238, 238, 241, 255, 259, 277].”
A White Paper on Neural Network Quantization (2021, background): “Homogeneous bit-width is more universally supported by hardware but some recent works also explore the implementation of heterogeneous bit-width or mixed-precision (van Baalen et al., 2020; Dong et al., 2019; Uhlich et al., 2020).”
Learned Token Pruning for Transformers (2021, result): “Our experimental results demonstrate that quantized LTP models achieve up to 10× reduction in the number of bit operations (BOPs) [43] with < 2% accuracy drop compared to the unpruned FP16 baseline.”
HAWQV3: Dyadic Neural Network Quantization (2020, background): “The latter measures the total Bit Operations for calculating a layer [48]:”
COIN: COmpression with Implicit Neural representations (2021, methodology): “However, we believe that more sophisticated approaches to architecture search [11] and especially model compression [36, 13, 37] will further improve results.”
Structured Pruning for Deep Convolutional Neural Networks: A Survey (2023, background): “Bayesian Bits (BB) [163] unifies the view of mixed precision quantization and pruning, and introduces mixed precision gates on filters for structured pruning.”