Original Contribution: Optimal filtering algorithms for fast learning in feedforward neural networks

By Samir Shah, Francesco Palmieri, M. Datum. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Local Gain Adaptation in Stochastic Gradient Descent (1999, methodology): “We evaluated our work on the \four regions" classiflcation task [27], a well-known benchmark problem [17, 28, 29 , 30]: a fully connected feedforward network with 2 hidden layers of 10 units each (tanh nonlinearity) is to classify two continuous inputs (range [-1,1]) as illustrated in Figure 1. We ”
Incremental Gradient Algorithms with Stepsizes Bounded Away from Zero (1998, background): “For many practical neural network systems, standard optimization methods require storage and/or computational cost which can become unmanageable even for a moderate network size, provided the training set (i.e., the number K ) is large enough [ 17 ].”
Centering Neural Network Gradient Factors (1996, background): “This is known to sometimes improve convergence behavior [ 11 ], and we have found it to increase stability at the large step sizes we desire.”