A Theoretical Framework for Back-Propagation

By Y. L. Cun. 1988

Subfields: Optimization & Training, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Statistical Learning Theory

Citations in the corpus

Neural Ordinary Differential Equations (2018, neutral): “†From LeCun et al. (1998).”
Deep learning in neural networks: An overview (2014, background): “Related work was published several years later (LeCun, 1985, 1988; Parker, 1985).”
Learning to compare image patches via convolutional neural networks (2015, background): “To that end, inspired also by the recent advances in neural architectures and deep learning, we choose to represent such a function in terms of a deep convolutional neural network [14, 13] (Fig.”
Can deep learning beat numerical weather prediction? (2021, methodology): “The invention of backpropagation in 1970 [5,6] led to a second”
Artificial Neural Networks for Neuroscientists: A Primer (2020, methodology): “For a feedforward network without any intermediate (hidden) layer [Rosenblatt, 1962], computing the gradient is straightforward, For a multi-layer network, the differentiation is done using the back-propagation algorithm [Rumelhart et al., 1988, LeCun, 1988].”
Training Deep Networks with Structured Layers by Matrix Backpropagation (2015, background): “The derivation of backpropagation[19] and its further advances more than a decade later [20, 21], allowed the development and the integration of new layers and the exploration of complex, more expressive architectures.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “However, it is only in recent years that learning algorithms are applied in wide domains, because of the progress of learning algorithms [16, 55, 61, 107] and computing power.”
RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr (2020, background): “From the error backpropagation’s perspectives (Rumelhart et al.; LeCun et al., 1988), the fast convergence of the FC layer leads to trivial weight updates to CNN (Liu et al.”
Gradient-based learning applied to document recognition (1998, methodology): “The best way to justify it theoretically is through the use of Lagrange functions [21], [22].”
A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex (2005, background): “, 2002; Thorpe, 2002; Amit and Mascaro, 2003] and biologically motivated computer vision approaches [Fukushima, 1980; Fukushima et al., 1983; Fukushima, 1986; LeCun, 1988; LeCun et al., 1998; Wersing and Koerner, 2003; LeCun et al., 2004].”
A Proposal on Machine Learning via Dynamical Systems (2017, background): “The connection to control theory has been explored in the work of [7], among others.”
Dendritic cortical microcircuits approximate the backpropagation algorithm (2018, background): “Interestingly, recent work has shown that backprop can be mapped onto a predictive coding network architecture (Whittington and Bogacz, 2017), related to the general framework introduced by LeCun (1988).”
Sim-to-real Transfer of Visuo-motor Policies for Reaching in Clutter: Domain Randomization and Adaptation with Modular Networks (2017, background): “From the backpropagation algorithm [25], we can infer that , where δ L is the gradients resulting from L ; δ L p and δ L BNc are the gradients resulting respectively from L p and L BNc (equivalent to that resulting from L c in the perception module).”