A Back-Propagation Algorithm with Optimal Use of Hidden Units

By Yves Chauvin. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, background): “The conventional approach to achieve sparsity is by enforcing penalty terms (Chauvin, 1989; Weigend et al., 1991; Ishikawa, 1996).”
Sparse Networks from Scratch: Faster Training without Losing Performance (2019, methodology): “Earlier work focused on pruning via second-order derivatives (LeCun et al., 1989; Karnin, 1990; Hassibi and Stork, 1992) and heuristics which ensure efﬁcient training of networks after pruning (Chauvin, 1988; Mozer and Smolensky, 1988; Ishikawa, 1996).”
Importance Estimation for Neural Network Pruning (2019, methodology): “We focus our attention on extending previously proposed methods [2, 22, 27] with a new pruning criterion and a method that iteratively removes the least important set of neurons (typically filters) from the trained model.”
SNIP: Single-shot Network Pruning based on Connection Sensitivity (2018, methodology): “The methods from the former category (Chauvin (1989); Weigend et al. (1991); Ishikawa (1996)) augment the loss function with some sparsity enforcing penalty terms ( e.g ., L 0 or L 1 norm), so that back-propagation effectively penalizes the magnitude of the weights during training.”
Constructive algorithms for structure learning in feedforward neural networks for regression problems (1997, methodology): “Early attempts either set this regularization parameter manually [1], [2] or by ad hoc procedures [3].”
Sparse evolutionary deep learning with over one million artificial neurons on commodity hardware (2019, methodology): “Motivated by these challenges, sparse neural networks (LeCun et al., 1990; Chauvin, 1989) have been introduced to effectively reduce the memory requirements to deploy ANN models.”
Pay Attention to Features, Transfer Learn Faster CNNs (2020, background): “Sparsity in neural networks has been a long-studied subject (Reed, 1993; LeCun et al., 1990; Chauvin, 1989; Mozer & Smolensky, 1989; Hassibi et al., 1994).”
Optimal Brain Damage (1989, methodology): “In fact, several network minimization schemes have been implemented using non-proportional weight decay (Rumelhart, 1988; Chauvin, 1989; Hanson and Pratt, 1989), or \gating coe cients" (Mozer and Smolensky, 1989).”
Generalization and network design strategies (1989, background): “…Several au thors have descnbed such schemes, usually Implemented as a non-proportional we1ght decay (Rumelhart, personnal communication 1988), (Chauvm, 1989, Hanson and Pratt, 1989), or usmg "gatmg coefficients" (Mozer and Smolensky, 1989) GeneralizatiOn performance has been reported to…”
A new strategy for adaptively constructing multilayer feedforward neural networks (2003, background): “This is a problem that has practical implications and has not received as much attention as necessary by neural networks researchers.”
The Effects of Adding Noise During Backpropagation Training on a Generalization Performance (1996, background): “This way of preventing overfitting is rather different from the way in which input noise prevents overfitting; it bears a resemblance to weight-decay methods (e.g., Chauvin 1989).”