AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters

By Xia Xiao, Zigeng Wang, Sanguthevar Rajasekaran. 2019

163 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

AutoML: A Survey of the State-of-the-Art (2019, background): “approaches have been proposed to compress the neural networks into more efficient forms [161, 162, 163].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “Pruning methods can be broadly categorized into unstructured pruning [49, 85, 137, 141, 186, 249], and structured pruning [90, 105, 153, 162, 266, 267, 271].”
Soft Threshold Weight Reparameterization for Learnable Sparsity (2020, background): “Concurrent to our work, (Savarese et al., 2019; Liu et al., 2020; Lee, 2019; Xiao et al., 2019) have proposed learnable sparsity methods through training of the sparse masks and weights simultaneously with minimal heuristics.”
Learned Token Pruning for Transformers (2021, methodology): “Pruning methods can be further categorized into unstructured pruning [12, 28, 33, 39, 54] and structured pruning [17, 18, 20, 29, 31, 49, 56, 57, 62].”
MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models (2021, background): “Pruning can be broadly categorized into unstructured pruning [9, 24, 49, 29, 15, 36] and structured pruning [26, 16, 51, 25, 18, 56, 52, 27].”
Distilling Knowledge via Knowledge Review (2021, methodology): “There have been techniques for training fast and compact neural networks, including designing new architectures [10, 2, 11, 26], network pruning [20, 15, 34, 4, 19], quantization [13] , and knowledge distillation [9, 25].”
Layer-adaptive Sparsity for the Magnitude-based Pruning (2020, methodology): “Other approaches include on-the-ﬂy 1 regularization (Louizos et al., 2018; Xiao et al., 2019), Bayesian methods (Molchanov et al., 2017; Louizos et al., 2017; Dai et al., 2018), and reinforcement learning (Lin et al., 2017).”