Using Relevance to Reduce Network Size Automatically

By Michael C. Mozer, Paul Smolensky. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

What is the State of Neural Network Pruning? (2020, methodology): “…with Han et al. (2015) reintroducing the magnitude-based pruning of Janowsky (1989), Lee et al. (2019b) reintroducing the saliency heuristic of Mozer & Smolensky (1989a), and He et al. (2018a) reintroducing the practice of “reviving” previously pruned weights described in Tresp et al. (1997).”
Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies (1997, unknown): “Network-shrinking algorithms start with too many parameters, then remove nodes and weights during training (Hassibi & Stork, 1992; Le Cun, Denker, & Solla, 1989; Mozer & Smolensky, 1989).”
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training (2021, methodology): “The criteria used for pruning includes but are not limited to magnitude (Mozer & Smolensky, 1989; Han et al., 2016; Guo et al., 2016), Hessian (LeCun et al., 1990; Hassibi & Stork, 1993), mutual information (Dai et al., 2018a), Taylor expansion (Molchanov et al., 2016; 2019).”
An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks (2004, methodology): “” The approximation for is , which fluctuates “strongly” in time [11], [12].”
Tuning of the structure and parameters of neural network using an improved genetic algorithm (2001, background): “Hidden layers, nodes, and connections are then deleted to contract the network dynamically [25], [26].”
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win (2020, methodology): “2 Related Work Pruning Pruning is used commonly in NN literature to obtain sparse networks (Mozer and Smolensky 1989; Han et al. 2015; Kusupati et al. 2020).”