Gradient descent learning algorithm overview: a general dynamical systems perspective

By Pierre Baldi. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “See generalizations for sequenceprocessing recurrent NNs (e.g., Atiya & Parlos, 2000; Baldi, 1995; Gherrity, 1989; Kremer & Kolen, 2001; Pearlmutter, 1989, 1995; Robinson & Fallside, 1987; Rohwer, 1989; Schmidhuber, 1992a; Werbos, 1988;Williams, 1989;Williams & Peng, 1990;Williams & Zipser, 1988,…”
A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training (2007, background): “Baldi [30] gave an overview about main learning algorithm based on gradient method, but the convergent rate was not studied.”
Machine Learning for Wireless Networks with Artificial Intelligence: A Tutorial on Neural Networks (2017, background): “From (3) and (4), we can see that, the update of wjk and bj is easy to compute and, hence, the gradient descent algorithm is known to be computationally fast, even on large datasets [75].”
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial (2017, background): “e E j(w j;b j). From (3) and (4), we can see that, the update of w jk and b j is easy to compute and, hence, the gradient descent algorithm is known to be computationally fast, even on large datasets [75]. The gradient descent algorithm mentioned above only focuses on the update of a single neuron. ”
Adaptive computation and machine learning (1998, methodology): “While the general gradient-descent principle is simple, in complex parameterized models it can give rise to different implementations, depending on how the gradient is actually computed [26].”
Adaptive stochastic resonance (1998, unknown): “Then di erentiate with respect to time [8] to get d dt = d dt @x @ = @ dx dt @ = @G(x; s; ; w) @ (107) = @G @x @x @ + @G @ = @G @x (t) + @G @ : (108) The last derivative @G @ results fromG's explicit dependence on .”
Prediction of contact maps by GIOHMMs and recurrent neural networks using lateral propagation from all four cardinal corners (2002, background): “Learning in neural-network-parameterized GIOHMMs can proceed by gradient descent for recurrent networks (Baldi, 1995), on typical error functions such as mean squared or relative entropy, by unfolding the structures through time or space.”
Assessing the accuracy of prediction algorithms for classification: an overview (2000, background): “In reality, it is not necessary that the network be layered as long as there are no directed loops—but layered networks are the most commonly used (see also Baldi, 1995).”
Fault Diagnosis for Rotating Machinery Using Multiple Sensors and Convolutional Neural Networks (2018, methodology): “Supervised training is performed using gradient descent, which can be implemented through the backpropagation algorithm [35].”