Pruning versus clipping in neural networks.

By S. A. Janowsky. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

On the Predictability of Pruning Across Scales (2020, methodology): “We study iterative magnitude pruning (IMP) (Janowsky, 1989; Han et al., 2015).”
What is the State of Neural Network Pruning? (2020, methodology): “Pruning has been used since the late 1980s (Janowsky, 1989; Mozer & Smolensky, 1989a;b; Karnin, 1990), but has seen an explosion of interest in the past decade thanks to the rise of deep neural networks.”
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training (2021, methodology): “Techniques sought to inherit weights from a fully pre-trained dense model have a long history and were first introduced by Janowsky (1989) and Mozer & Smolensky (1989), autonomously evolving as the iterative pruning and retaining method.”
Pruning neural networks without any data by iteratively conserving synaptic flow (2020, background): “Network pruning, or the compression of neural networks by removing parameters, has been an important subject both for reasons of practical deployment [1, 2, 3, 4, 5, 6, 7] and for theoretical understanding of artificial [8] and biological [9] neural networks.”
Layer-adaptive Sparsity for the Magnitude-based Pruning (2020, methodology): “Magnitude-based pruning of neural networks dates back to the early works of Janowsky (1989); LeCun et al. (1989), and has been actively studied again under the context of model compression since the work of Han et al. (2015).”