An empirical study of learning speed in back-propagation networks

By Scott E. Fahlman. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “To speed up BP, momentum was introduced (Rumelhart et al., 1986), ad-hoc constants were added to the slope of the linearized activation function (Fahlman, 1988), or the nonlinearity of the slope was exaggerated (West & Saad, 1995).”
Neural Networks in R Using the Stuttgart Neural Network Simulator: RSNNS (2012, background): “The function used earlier as an example, SnnsCLib__setLearnFunc , can be called like this: snnsObject$setLearnFunc("Quickprop") The $ operator ﬁrst searches an R function with the name SnnsR__setLearnFunc .”
G-Prop-III: Global Optimization of Multilayer Perceptrons using an Evolutionary Algorithm (1999, methodology): “The tests used to assess the accuracy of both methods must be chosen carefully, because some tests (exclusive-or problem) are not suitable for certain capacities of the BP algorithm, such as generalization [30].”
Convexified Convolutional Neural Networks (2016, background): “This approach has two drawbacks: (i) the rate of convergence, which is at best only to a local optimum, can be slow due to nonconvexity (for instance, see the paper [19]), and (ii) its statistical properties are very difficult to understand, as the actual performance is determined by some combinatio”
Early Stopping-But When? (1996, background): “RPROP is a fast backpropagation variant that is about as fast as quickprop [6] but more stable without adjustment of the parameters.”
Symbiotic Evolution of Neural Networks in Sequential Decision Tasks (1997, methodology): “Potter (1992) used an evolutionary algorithm in place of the quickprop learning method (Fahlman 1988) in the cascade correlation architecture (Fahlman and Lebiere 1990), which is one 24”
When Does Label Smoothing Help? (2019, background): “However, even in these early days of neural network research, there were indications that other, more exotic objectives could outperform the standard cross entropy loss [4, 5].”
Metaheuristic design of feedforward neural networks: A review of two decades of research (2017, methodology): “Since the basic version of BP is sensitivity towards learning rate and momentum factor [15], several improvements were suggested by researchers: 1) a fast BP algorithm, called Quickpro was proposed in [66, 67]; 2) a delta-bar technique and an acceleration technique was suggested for tuning BP learni”
A direct adaptive method for faster backpropagation learning: the RPROP algorithm (1993, methodology): “The first problem to be described is the 10-5-10 Encoder task, for it is also discussed largely in [ 5 ].”
Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks (2007, background): “This is quite close to real world pattern classification tasks, where small changes in the input pattern cause small changes in the output pattern [37].”
Accelerated Learning in Back-propagation Nets (1989, methodology): “Fahlman's method (Fahlman, 1988), for instance, estimates the curvature of the error surface by using rst order information gained from each weight during successive time ticks.”