A Focused Backpropagation Algorithm for Temporal Pattern Recognition

By Michael C. Mozer. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Lifelong learning of human actions with deep neural network self-organization (2017, methodology): “…is represented by the current network parameters and the prediction error is used to compute the loss cost from ground-truth sample labels, and (ii) a backward stage which computes the gradients with respect to the parameters and updates themusing back-propagation through time (BPTT, Mozer, 1995).”
Crafting adversarial input sequences for recurrent neural networks (2016, methodology): “We leverage the technique known as computational graph unfolding [18], [19] to compute the forward derivative in the presence of cycles, as is the case with RNNs.”
Strategic Attentive Writer for Learning Macro-Actions (2016, methodology): “We cut the trajectory and run backpropagation through time [19] after 40 forward passes of a network or if a terminal signal is received.”
Natural Language Grammatical Inference with Recurrent Neural Networks (2000, result): “The FGS network has also been studied by [43] – the network is called FGS in this paper in line with [63].”
The Induction of Dynamical Recognizers (1991, background): “, 1986), involves a complete unrolling of a recurrent loop and has had only modest success (Mozer, 1988), probably because of conflicts arising from equivalence constraints between interdependent layers.”
Recurrent Convolutional Network for Video-Based Person Re-identification (2016, methodology): “During training with back propagation through time, all recurrent connections are unrolled to create a deep feed-forward graph, where the weights of the recurrent layer and CNN are shared between all timesteps [31].”
Long short-term memory networks in memristor crossbar arrays (2018, methodology): “The intended conductance update values ( ∆ G ) were calculated using the back-propagation through time (BPTT) algorithm\cite{sgdm, mozer1989bptt, werbos1988bptt} (see Methods for details).”
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation (2015, methodology): “During the back-propagation stage, DRNN uses back-propagation through time (BPTT) [29], [30] to update network parameters.”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, methodology): “RNNs are trained using the backpropagation through time algorithm (Mozer, 1989; Werbos, 1990), which applies backpropagation after first unrolling the network.”
Induction of Multiscale Temporal Structure (1991, methodology): “Time constants have been incorporated into the activation rules of other connectionist architectures (Jordan, 1987; McClelland, 1979; Mozer, 1989; Pearlmutter,1989; Pineda, 1987).”
On the Origin of Deep Learning (2017, methodology): “This solution is known as Backpropagation through Time (BPTT), independently invented by several researchers including (Robinson and Fallside, 1987; Werbos, 1988; Mozer, 1989) However, as recurrent neural network usually has a more complex cost surface, naive backpropagation may not work well.”
Neural net architectures for temporal sequence processing (2007, methodology): “Note that this architecture is quite similar to a fairly common recurrent neural net sequence processing architecture (Figure 4b; e.g., Elman, 1990; Mozer, 1989), which I’ll refer to as the standard architecture.”
On-line learning algorithms for locally recurrent neural networks (1999, methodology): “In fact they are very general architectures which can model a large class of dynamical systems, but on specific problems simpler dynamic neural networks which make use of available prior knowledge can be better [21], [41].”