Gradient-based learning algorithms for recurrent connectionist networks

By Ronald J. Williams, David Zipser. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

DDSP: Differentiable Digital Signal Processing (2020, background): “…in the asymptotic limit (Hornik et al., 1989), but their practical success is largely due to the use of strong structural priors such as convolution (Le-Cun et al., 1989), recurrence (Sutskever et al., 2014; Williams & Zipser, 1990; Werbos, 1990), and self-attention (Vaswani et al., 2017).”
An application of recurrent nets to phone probability estimation (1994, background): “[1, 2, 3]), however this paper is interested in the kind that map one sequence on to another.”
An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories (1990, methodology): “Studies reported in Williams and Zipser (1990), in which RTRL was compared with BPTT(h), are also consistent with this result.”
A Novel Connectionist System for Unconstrained Handwriting Recognition (2009, methodology): “The network can be trained with gradient descent by first differentiating O with respect to the outputs and then using backpropagation through time [47] to find the derivatives with respect to the network weights.”
Training recurrent networks using the extended Kalman filter (1992, background): “The relationships be-tween these and several variants, all based on compu-tation of the gradient of an output error measure withrespect to network weights, are discussed at lengthin Williams and Zipser (1990).”
Framewise phoneme classification with bidirectional LSTM networks (2005, unknown): “The original LSTM training algorithm [11] used an error gradient calculated with a combination of Real Time Recurrent Learning (RTRL)[20] and Back Propagation Through Time (BPTT)[24].”
Bifurcations of Recurrent Neural Networks in Gradient Descent Learning (1993, background): “One reason for this is the large cost for gradient computation [26].”