Evaluating pruning methods

By G. Thimm, E. Fiesler. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints (2022, methodology): “Magnitude-based methods [41, 18] perform one or more rounds of pruning, by removing the parameters with the lowest magnitudes.”
Rigging the Lottery: Making All Tickets Winners (2019, methodology): “Research on ﬁnding sparse neural networks dates back decades, at least to Thimm & Fiesler (1995) who concluded that pruning weights based on magnitude was a simple and powerful technique.”
TIES-Merging: Resolving Interference When Merging Models (2023, background): “…merged model and eliminate subtle distinctions among values: (1) I NTERFERENCE FROM REDUNDANT PARAMETERS : Previous studies on model pruning [25, 76] have shown that during fine-tuning, many model parameters can change over the course of fine-tuning [63] but only have a small impact on…”
Channel-Level Acceleration of Deep Face Representations (2015, methodology): “Originally, [22] define scores for the contribution of a single weight to the activation of a single neuron.”
Loss of plasticity in deep continual learning (2024, unknown): “On the other hand, PPO augmented with”
Sparse connection and pruning in large dynamic artificial neural networks (1997, methodology): “An overview of this and other methods is found in [8].”
Fast Sparse ConvNets (2019, background): “While the idea of using sparsity to decrease the parameter count is not new [43], the conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains.”