Theory of the backpropagation neural network

By Robert Hecht-Nielsen. 1989

Subfields: Model Architectures, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Optimization & Training

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “Many practitioners found solace in a theorem (Hecht-Nielsen, 1989; Hornik, Stinchcombe, & White, 1989; Kolmogorov, 1965a) stating that an NNwith a single layer of enough hidden units can approximate anymultivariate continuous function with arbitrary accuracy.”
Recent advances in convolutional neural networks (2015, background): “It can obtain effective representations of the original image, which makes it possible to recognize visual patterns directly from raw pixels with little-to-none preprocessing.”
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence (2018, background): “A better strategy would be to exploit the over-parametrization of neural networks [3].”
Active Long Term Memory Networks (2016, background): “Generally, in the sequential estimation case, the most recent task benefits from an inductive bias [14, 11] while older tasks become distorted [15, 16, 17] by unconstrained back-propagation [18, 19] of errors in shared parameters, a problem identified as Catastrophic Interferences (CI) between tasks”
Fixed-point feedforward deep neural network design using weights +1, 0, and −1 (2014, methodology): “The mini-batch based backpropagation algorithm [14] updates the weight wij , the synaptic strength from the unit j to the unit i, by”
Machine Learning Algorithms in Civil Structural Health Monitoring: A Systematic Review (2020, background): “The weight that minimizes the loss function is the solution for the learning problem [50].”
Stochastic Configuration Networks: Fundamentals and Algorithms (2017, background): “OVER the past decades, neural networks have been successfully applied for data modeling due to its universal approximation power for nonlinear maps [1]–[4], and learning capability from a collection of training samples [5], [6].”
A Sparse-View CT Reconstruction Method Based on Combination of DenseNet and Deconvolution (2018, methodology): “Once the architecture of the network is ﬁxed, all the parameters can be estimated using backpropagation algorithm (BP) [51] by optimizing a loss function.”
Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network (2015, unknown): “The process of fine-tuning is back-propagation, searching for a minimum in a peripheral region of parameters initialized by DBN [47], [48].”
A neural network approach for attenuation relationships: An application using strong ground motion data from Turkey (2007, background): “…seems to create some doubts about the generalization capability of the ANN models, it should be noted that one hidden layer of their architectures can be considered small for a network with three input variables, which improves the generalization of the models as discussed by Hecht-Nielsen (1989).”
Weighted residuals for very deep networks (2016, background): “where the convex set S = (−1, 1) and Δλi is the gradient of the loss function in Equation 4 with regard to λi, which is effectively computed by back-propagation [18] in deep networks.”