The truck backer-upper: an example of self-learning in neural networks

By Derrick Nguyen, Bernard Widrow. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, background): “Many basic concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) [56, 58, 72, 104, 105] and in the 1990s for RNNs [74, 75, 76, 78] laid some of the groundwork for Learning to Think [83].”
Causally Correct Partial Models for Reinforcement Learning (2020, methodology): “One approach to capture this ability is via model-based reinforcement learning (MBRL) (Munro, 1987; Werbos, 1987; Nguyen & Widrow, 1990; Schmidhuber, 1991).”
Shaping Belief States with Generative Environment Models for RL (2019, background): “The idea of learning general world models to support decision-making is probably one of the most pervasive ideas in AI research, [30, 11, 41, 42, 14, 43, 44, 45, 46, 33, 30, 47, 48, 3, 49, 50, 51, 52].”
World Models (2018, background): “Many concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) (Werbos, 1987; Munro, 1987; Robinson & Fallside, 1989; Werbos, 1989; Nguyen & Widrow, 1989) and in the 1990s for RNNs (Schmidhuber, 1990a;b; 1991a; 1990c) laid some of the groundwork for Learning to Think…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Today’s faster computers, such as GPUs, mitigate the Fundamental Deep Learning Problem for FNNs [181, 34, 198, 38, 40].”
Optimal Ordered Problem Solver (2002, methodology): “The method was inspired by previous combinations of nonrecurrent, reactive world models and controllers (Werbos, 1987, Nguyen and Widrow, 1989, Jordan and Rumelhart, 1990).”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “Using SL for certain aspects of RL dates back to the 1980s and 90s [70, 31, 22, 73, 72, 39, 32].”
Exploring Deep and Recurrent Architectures for Optimal Control (2013, methodology): “Early experiments with neural network control represented both the system dynamics and policy as neural networks, so that the gradient of the policy could be propagated backwards in time [18, 6].”
Learning to Control an Unstable System with Forward Modeling (1989, methodology): “In this section we briefly summarize our previous work on forward modeling (see also Nguyen & Widrow, 1989 and Werbos, 1987).”
New results on recurrent network training: unifying the algorithms and accelerating convergence (2000, background): “…or finite impulse response (FIR) networks [39]–[41], etworks with trainable delays [42], networks [43], memory neuron networks [44], generalized recursive neuron networks [45], [46], etc, have shown much success, and can be trained with some modifications of existing recurrent network algorithms.”
Curious model-building control systems (1991, background): “For instance, [2], [ 3 ], [IO], and others use random search to train the world model.”
Making the World Di erentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning (1990, background): “A similar o -line approach (without internal feedback) also was described in Nguyen and Widrow's work (see [16] for an interesting application that does not require internal feedback).”
Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (1992, methodology): “The backpropagation of utility, which maximizes some measure of utility or performance over time, but cannot efficiently account for noise and cannot provide real-time learning for very large problems [3] [4] [5] .”
An on-line algorithm for dynamic reinforcement learning and planning in reactive environments (1990, background): “Nguyen and Widrow [ 4 ], Jordan [2], and Werbos [ll] a.lso use model networks for constructing a mapping from output actions of a control network to their effects in in ‘task space’ [2].”
Deep learning in neural networks: An overview (2014, methodology): “Decades ago, the method was used to learn to back up a model truck (Nguyen and Widrow, 1989).”