Neural networks for control and system identification

By Paul J. Werbos. 1989

Subfields: Optimization & Training, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Robotics & Control

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, background): “Many basic concepts first explored in the 1980s for feed-forward neural networks (FNNs) [56, 58, 72, 104, 105] and in the 1990s for RNNs [74, 75, 76, 78] laid some of the groundwork for Learning to Think [83].”
Deep learning in neural networks: An overview (2014, neutral): “…& Kawato, 1993; Jordan, 1988; Jordan & Rumelhart, 1990; Levin & Narendra, 1995; Ljung, 1998; Miller, Werbos, & Sutton, 1995; Munro, 1987; Narendra & Parthasarathy, 1990; Prokhorov, Puskorius, & Feldkamp, 2001; Robinson & Fallside, 1989; Schmidhuber, 1990d; Werbos, 1981, 1987, 1989a, 1989b, 1992).”
World Models (2018, background): “Many concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) (Werbos, 1987; Munro, 1987; Robinson & Fallside, 1989; Werbos, 1989; Nguyen & Widrow, 1989) and in the 1990s for RNNs (Schmidhuber, 1990a;b; 1991a; 1990c) laid some of the groundwork for Learning to Think…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1) could in principle plan ahead by performing numerous fast mental experiments on a predictive RNN world model, M , instead of time-consuming real experiments, extending earlier work on reactive systems without memory [301, 273].”
Model-free Q-learning designs for discrete-time zero-sum games with application to H-infinity control (2007, methodology): “Q-function setup for discrete-time linear quadratic zero-sum games installation In this section, we formulate Bellman’s optimality principle for the zero-sum-game using the concept of Q-functions (Watkins, 1989; Werbos, 1990) instead of the standard value functions used elsewhere.”
Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems (2009, unknown): “…of using RL for solving optimal control problems comes from the fact that a number of RL algorithms, e.g. Q-learning (Watkins, 1989) (also known as Action Dependent Heuristic Dynamic Programming (Werbos, 1989, 1992)), do not require knowledge or identification/learning of the system dynamics.”
Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (1992, methodology): “In 1989, Lukes, Thompson, and myself [38,39] and Watkins [40] independently arrived at methods to adapt action-dependent critics, shown in Figure 3.”
Adaptive optimal control for continuous-time linear systems based on policy iteration (2009, unknown): “They gave a proof of convergence for Q-learning policy iteration for discrete-time systems, which, by virtue of using the so called Q-functions (Watkins, 1989; Werbos, 1989), does not require any knowledge of the system dynamics.”
Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning (2014, methodology): “The learning structure uses the value function approximation (Finlayson, 1990; Werbos, 1989, 1992) with two NNs, namely an actor and a critic.”
Reinforcement Learning in Continuous State and Action Spaces (2012, methodology): “Both approaches have often been used in reinforcement learning with considerable success (Sutton, 1988; Werbos, 1989b,a, 1990; Whitley et al, 1993; Tesauro, 1994, 1995; Moriarty andMiikkulainen, 1996; Moriarty et al, 1999; Whiteson and Stone, 2006; Wierstra et al, 2008; Rückstieß et al, 2010).”
Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem (2009, methodology): “Actor/critic structures based on Value Iteration have been introduced and further developed by Werbos [19], [20], [21] with the purpose of solving the optimal control problem online in real-time.”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “Using SL for certain aspects of RL dates back to the 1980s and 90s [72, 32, 23, 75, 74, 40, 33].”
Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers (2012, methodology): “A Q learning algorithm is easily developed for discrete-time dynamical systems using Q function approximation [7], [14], [19], [49].”
Sequential Constant Size Compressors for Reinforcement Learning (2011, methodology): “Gradientbased RL based on interacting RNNs [20] extend Werbos’ work based on feedforward nets [24]: One RNN (the model net) serves to model the environment,”
Reinforcement learning and adaptive dynamic programming for feedback control (2009, methodology): “It is shown in [Vrabie 2009] that the nonlinear Lyapunov equation (85) is exactly equivalent to the interval reinforcement form (89).”