Mixed Precision DNNs: All you need is a good parametrization

165 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “, 2021), and differentiable quantization methods (Uhlich et al., 2020; Wang et al., 2020).”
Overcoming Oscillations in Quantization-Aware Training (2022, methodology): “In practice, the former is the same as setting the gradient outside the quantization grid to zero, as per equation (2), and is the most widely used formulation in QAT literature (Choi et al., 2018; Esser et al., 2020; Uhlich et al., 2020).”
Bayesian Bits: Unifying Quantization and Pruning (2020, methodology): “The drawback, similarly to Uhlich et al. (2020), of such an approach is that there is no inherent control over the resulting bit widths.”
NIPQ: Noise Injection Pseudo Quantization for Automated DNN Optimization (2022, methodology): “Various methods, such as RL-based [28, 29, 30], Hessian-based [31, 32, 33], and differentiable [34, 16] methods, have been proposed.”
A White Paper on Neural Network Quantization (2021, background): “Homogeneous bit-width is more universally supported by hardware but some recent works also explore the implementation of heterogeneous bit-width or mixed-precision (van Baalen et al., 2020; Dong et al., 2019; Uhlich et al., 2020).”
LSQ+: Improving low-bit quantization through learnable offsets and better initialization (2020, background): “Also, some recent work [28] has been done on automatically learning the bit-width alongside of the ranges.”