The "Moving Targets" Training Algorithm

By Richard James Rohwer. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “…recurrent NNs (e.g., Atiya & Parlos, 2000; Baldi, 1995; Gherrity, 1989; Kremer & Kolen, 2001; Pearlmutter, 1989, 1995; Robinson & Fallside, 1987; Rohwer, 1989; Schmidhuber, 1992a; Werbos, 1988;Williams, 1989;Williams & Peng, 1990;Williams & Zipser, 1988, 1989a, 1989b), also for equilibrium RNNs…”
Distributed optimization of deeply nested systems (2012, background): “Interest in using the activations of neural nets as independent variables goes back to the early days of neural nets, where learning good internal representations was as important as learning good weights (Grossman et al., 1988; Saad and Marom, 1990; Krogh et al., 1990; Rohwer, 1990).”
A Connectionist Symbol Manipulator that Discovers the Structure of Context-Free Languages (1992, background): “The di cult problem of extracting multilevel structure from complex, extendedsequences has been studied by Mozer (1992), Ring (1993), Rohwer (1990), andSchmidhuber (1992), among others.”
Learning long-term dependencies with gradient descent is difficult (1994, methodology): “Manuscript received April 21 , 1993; revised December 21, 1993.”
Induction of Multiscale Temporal Structure (1991, background): “In practice, however, back propagation is not su ciently powerful to discover arbitrary contingencies, especially those that span long temporal intervals or that involve high order statistics (e.g., Mozer, 1989; Rohwer, 1990; Schmidhuber, 1991).”
Gradient-based learning algorithms for recurrent networks and their computational complexity (1995, methodology): “Finally, we note that Rohwer (1990) has expanded on this idea of teacher forcing to develop an interesting new epochwise learning algorithm based on computation of the gradient of performance with respect to unit activities rather than network weights.”
Gradient-based learning algorithms for recurrent connectionist networks (1990, background): “…from later times by the factor 1 in BPTT or multiplying plij(t) by 1 beforepropagating the activity gradient forward in RTRL.Finally, we note that Rohwer (1989) has expanded on this idea of teacher forcing to develop aninteresting new epochwise learning algorithm based on computation of the…”