Pruning and Quantization for Deep Neural Network Acceleration: A Survey
By Tailin Liang, C. John Glossner, Lei Wang, Shaobo Shi. 2021
981 citations recorded by the source corpus.
Abstract Deep neural networks have been applied in many applications exhibiting extraordinary abilities in the field of computer vision. However, complex network architectures challenge efficient real-time deployment and require significant computation resources and energy costs. These challenges can be overcome through optimizations such as network compression. Network compression can often be realized with little loss of accuracy. In some cases accuracy may even improve. This paper provides a survey on two types of network compression: pruning and quantization. Pruning can be categorized as static if it is performed offline or dynamic if it is performed at run-time. We compare pruning techniques and describe criteria used to remove redundant computations. We discuss trade-offs in element-wise, channel-wise, shape-wise, filter-wise, layer-wise and even network-wise pruning. Quantization reduces computations by reducing the precision of the datatype. Weights, biases, and activations may be quantized typically to 8-bit integers although lower bit width implementations are also discussed including binary neural networks. Both pruning and quantization can be used independently or combined. We compare current techniques, analyze their strengths and weaknesses, present compressed network accuracy results on a number of frameworks, and provide practical guidance for compressing networks.
Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition, Efficiency & Compression
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computational Mechanics
In Computer Vision and Pattern Recognition
In Efficiency & Compression
Citations in the corpus
- Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch (2023, positive): “As long as a small portion of the delta parameters remain unaffected during merging, the abilities of LMs unlocked by SFT can still be preserved.”
- Structured Pruning for Deep Convolutional Neural Networks: A Survey (2023, methodology): “In addition, the quantization process also includes clustering and parameter sharing [40], and quantization can be combined with pruning [1], [284], [285].”
- F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization (2022, background): “…and extensively studied in the last decades, resulting in significant improvement on the trade-off between efficiency and accuracy (Han et al., 2015; Liu et al., 2018; Mei et al., 2019; Tanaka et al., 2020; Ma et al., 2020; Mishra et al., 2020; Liang et al., 2021; Jin et al., 2021; Liu et al., 2021).”
- DepGraph: Towards Any Structural Pruning (2023, background): “Among the many network compression paradigms, pruning has proven itself to be highly effective and practical [7, 11, 30, 31, 44, 58, 59, 74].”
- Structured Pruning Adapters (2022, neutral): “Quantization approaches (Gray & Neuhoff, 1998; Liang et al., 2021) reduce model size and run-time costs via low-resolution numerical representations of network weights.”
- Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (2022, positive): “An alternative but challenging scenario is the post-training compression setup [31, 24, 19, 25], in which we are given a trained but uncompressed model, together with a small amount of calibration data , and must produce an accurate compressed model in one shot , i.e. a single compression step,…”