MetaQuant: Learning to Quantize by Learning to Penetrate Non-differentiable Quantization

By Shangyu Chen, Wenya Wang, Sinno Jialin Pan. 2019

53 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “Meta-Quant (Chen et al., 2019) achieves smaller model size than DiffQ, with 1 bit per weight, a regime where the PQN assumption breaks down, at the price of losing nearly 10% of accuracy. Finally, compared with two quantization methods: DQ by Uhlich et al. (2020) and LSQ by Esser et al.”
Meta-Learning in Neural Networks: A Survey (2020, background): “Meta-learning is beginning to be applied to this objective as well, such as training gradient generator meta-networks that allow quantized networks to be trained [187], and weight generator meta-networks that allow quantized networks to be trained with gradient [279].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “While STE is the mainstream approach [218, 280], other approaches have also been explored in the literature [2, 25, 31, 59, 142, 160].”
Overcoming Oscillations in Quantization-Aware Training (2022, background): “2019) approximate the gradient using a small neural network, whose parameters are learnt using meta-learning but is limited to binary weights.”
Binary Neural Networks: A Survey (2020, unknown): “In the literature, approaches for compressing the deep networks can be classified into five categories: parameter pruning [26, 29, 30, 31], parameter quantizing [32, 33, 34, 35, 36, 37, 38, 39, 40, 41], low-rank parameter factorization [42, 43, 44, 45, 46], transferred/compact convolutional filters ”