Dynamic reinforcement driven error propagation networks with application to game playing

By Tony A. Robinson, F. Fallside. 1989

Subfields: Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, background): “Many basic concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) [56, 58, 72, 104, 105] and in the 1990s for RNNs [74, 75, 76, 78] laid some of the groundwork for Learning to Think [83].”
Deep learning in neural networks: An overview (2014, background): “lled M can learn to become C’s world model through system identiﬁcation , predicting C’s inputs from previous actions and inputs (e.g., Werbos, 1981, 1987; Munro, 1987; Jordan, 1988; Werbos, 1989b,a; Robinson and Fallside, 1989; Jordan and Rumelhart, 1990; Schmidhuber, 1990d; Narendra and Parthasara”
RUDDER: Return Decomposition for Delayed Rewards (2018, neutral): “However sensitivity analysis by “backpropagation through a model” 58 [35, 44, 45, 4] has major drawbacks: local minima, instabilities, exploding or vanishing gradients in 59 the world model, proper exploration, contribution (relevance) of actions are not regarded only their 60 sensitivity [18, 48].”
World Models (2018, background): “Many concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) (Werbos, 1987; Munro, 1987; Robinson & Fallside, 1989; Werbos, 1989; Nguyen & Widrow, 1989) and in the 1990s for RNNs (Schmidhuber, 1990a;b; 1991a; 1990c) laid some of the groundwork for Learning to Think…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The first combination of an RL RNN C and an UL RNN M was ours and dates back to 1990 [223, 222, 226, 227], generalizing earlier similar controller/model systems (CM systems) based on FNNs [298, 179]; compare related work [177, 119, 301, 300, 209, 120, 178, 302, 73, 45, 144, 166, 153, 196, 60][245, S”
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019, background): “Providing information to an agent empowers them [61], hence you should first be confident that their interests align with yours.”
Learning Algorithms for Networks with Internal and External Feedback (1990, methodology): “In diierent contexts and with diierent degrees of generality, this basic principle for credit assignment based on system identiication has been previously described in (Werbos, 1977), (Jordan, 1988), (Munro, 1987), (Robinson and Fallside, 1989), (Nguyen and Widrow, 1989), and (Schmidhuber, 1990b).”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “Using SL for certain aspects of RL dates back to the 1980s and 90s [70, 31, 22, 73, 72, 39, 32].”
Reinforcement Learning in Markovian and Non-markovian Environments (1991, unknown): “…(cid:152)cY(cid:26)(cid:130)(cid:15)jcVA(cid:144)Li(cid:146)b}tvz’j}(cid:128)m]ƒbcV-]‘VGwvuxt(cid:136)(cid:133)§bc(cid:129){u{~E£puxwL¢7u{u{V-(cid:132)‘u(cid:6)b3tvVNuxt7bcVNl 32 ^(cid:230)¥(cid:221)rº{Ø#ª›Ø}(cid:230)¥(cid:221)cŒ3(cid:223)GªxºxØ (cid:11)4 /Ø3(cid:221)^Ł 65 (cid:10)ª›(cid:226) )7…”
Forward Models: Supervised Learning with a Distal Teacher (1992, background): “There have been a number of further developments of the idea (Kawato, 1990; Miyata, 1988; Munro, 1987; Nguyen & Widrow, 1989; Robinson & Fallside, 1989; Schmidhuber, 1990), based on either on the work of Werbos or our own unpublished work (Jordan, 1983; Rumelhart, 1986).”
Explaining and Interpreting LSTMs (2019, methodology): “lready used in reinforcement learning [66] for advantage learning [7] and learning policies [27,50,28]. However, backward analysis via sensitivity analysis like \backpropagation through a model" [55,60,61,8] have major drawbacks: local minima, instabilities, exploding or vanishing gradients in ”
Learning to Generate Artificial Fovea Trajectories for Target Detection (1991, background): “In di erent contexts and with di erent degrees of generality the basic principle for credit assignment by system realization and `gradient descent through a frozen model network' has been previously described by Werbos [19], Jordan [2], Munro [5], Robinson & Fallside [8], Nguyen & Widrow [6], and Sc”
A possibility for implementing curiosity and boredom in model-building neural controllers (1991, background): “Munro [2], Jordan [1], Werbos [12], Robinson and Fallside [6], and Nguyen and Widrow [4] used `model networks' for constructing a mapping from output actions 2”