Accelerated Backpropagation Learning: Two Optimization Methods

By Roberto Battiti. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “Some algorithms for controlling BP step size adapt a global learning rate (Battiti, 1989; Lapedes & Farber, 1986; LeCun, Simard, & Pearlmutter, 1993; Vogl, Mangis, Rigler, Zink, & Alkon, 1988; Yu, Chen, & Cheng, 1995), while others compute individual learning rates for each weight (Jacobs, 1988;…”
Neural networks for the prediction and forecasting of water resource variables: a review of modelling issues and applications (2000, methodology): “The methods proposed by Lapedes and Farber (1986); Battiti (1989); Jacobs (1988) fall in the above category.”
Centering Neural Network Gradient Factors (1996, background): “In a batch implementation this leaves only one free parameter to be determined: the global learning rate ·. Bold driver [14, 15, 16 , 17].”
Partial BFGS Update and Efficient Step-Length Calculation for Three-Layer Neural Networks (1997, methodology): “In order to cope with this problem, the OSS algorithm ( Battiti 1989 ) adopted the memoryless update (Gill et al. 1981); however, OSS may not provide desirable performance because the descent direction is calculated on the basis of a fast but rough approximation.,In order to cope with this problem, ”
Recommender Systems (2016, unknown): “Alternatively, one might use the bold driver algorithm [58, 217] to select α adaptively in each iteration in order to avoid local optima and speed up convergence.”
Hamming Distance Metric Learning (2012, methodology): “We use a heuristic to adapt learning rates, known as bold driver [2].”
Learning Rate Adaptation in Stochastic Gradient Descent (2001, methodology): “ADAPTATION Despite the abundance of methods for learning from examples, there are only few that can be used effectively for on–line learning.”
Local Gain Adaptation in Stochastic Gradient Descent (1999, background): “Unfortunately most of the existing gain adaptation algorithms for neural networks adapt only a single, global learning rate [6, 7], can be used only in batch training [8, 9, 10, 11, 12, 13], or both [14, 15, 16, 17].”
Neural networks for pattern recognition (1995, methodology): “One such approach is the bold driver technique (Vogl et al., 1988; Battiti, 1989).”
Perceptrons (2021, methodology): “One of these automatic approaches is called bold driver technique[43, 3].”
Neural Networks: Tricks of the Trade (2012, methodology): “On the other hand, weight decay regularizers [5, 2] are well understood, but finding a suitable parameter λ to control the strength of the weight decay term can be tediously time consuming.”