Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research

By Paul J. Werbos. 1987

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “…& Kawato, 1993; Jordan, 1988; Jordan & Rumelhart, 1990; Levin & Narendra, 1995; Ljung, 1998; Miller, Werbos, & Sutton, 1995; Munro, 1987; Narendra & Parthasarathy, 1990; Prokhorov, Puskorius, & Feldkamp, 2001; Robinson & Fallside, 1989; Schmidhuber, 1990d; Werbos, 1981, 1987, 1989a, 1989b, 1992).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Today’s faster computers, such as GPUs, mitigate the Fundamental Deep Learning Problem for FNNs [181, 34, 198, 38, 40].”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “RL emerged as a tool in the artificial intelligence (AI) and neural research communities, where combining DP with derivative-based adaptive function approximations (Werbös, 1987) and learning-based methods (Barto et al., 1983) was advocated in the mid-1980s.”
Forward Models: Supervised Learning with a Distal Teacher (1992, methodology): “The idea of using an internal model to augment he capabilities of supervised learning algorithms was also proposed by Werbos (1987), although his perspective differs in certain respects from our own.”
Adaptive Critic Nonlinear Robust Control: A Survey (2017, background): “As Werbos [205]–[208] pointed out, ADP may be the only approach that can achieve truly brain-like intelligence.”
Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (1992, background): “Globalized DHP (GDHP), which adapts a Critic network whose output is an approximation of J(R(t», but adapts it so as to minimize errors in the implied derivatives of J (as well as J itself, with some weighting) [35,36,37].”
Reinforcement Learning: An Introduction (1998, methodology): “Werbos (1977) suggested an approach to approximating DP called “heuristic dynamic programming” that emphasizes gradient-descent methods for continuous-state problems (Werbos, 1982, 1987, 1988, 1989, 1992).”
Linear Least-Squares algorithms for temporal difference learning (2004, methodology): “The LS TD algorithm has some similarity to an algorithm Werbos (Werbos, 1990) proposed as a linear version of his Heuristic Dynamic Programming (Lukes, et al., 1990, Werbos, 1987, Werbos, 1988, Werbos, 1992).”
An upper bound on the loss from approximate optimal-value functions (1994, background): “…C. YeeDepartment of Computer ScienceUniversity of MassachusettsAmherst, MA 01003(413)-545-1596singh@cs.umass.edu, yee@cs.umass.eduAbstractMany reinforcement learning (RL) approaches can be formulatedfrom the theory of Markov decision processes and the associatedmethod of dynamic programming (DP).”
Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming (1990, unknown): “Werbos (1987) has previously argued for the general idea of building AI systems that approximate dynamic programming, and Whitehead (1989) and others (Sutton & Barto, 1981; Sutton & Pinette, 1985; Rumelhart et al., 1986) have presented results for the speciic idea of augmenting a reinforcement…”
Reinforcement Learning in Markovian and Non-markovian Environments (1991, unknown): “…(cid:152)cY(cid:26)(cid:130)(cid:15)jcVA(cid:144)Li(cid:146)b}tvz’j}(cid:128)m]ƒbcV-]‘VGwvuxt(cid:136)(cid:133)§bc(cid:129){u{~E£puxwL¢7u{u{V-(cid:132)‘u(cid:6)b3tvVNuxt7bcVNl 32 ^(cid:230)¥(cid:221)rº{Ø#ª›Ø}(cid:230)¥(cid:221)cŒ3(cid:223)GªxºxØ (cid:11)4 /Ø3(cid:221)^Ł 65…”
Adaptive Dynamic Programming for Control: A Survey and Recent Advances (2021, background): “In terms of control applications, the term approximate dynamic programming for optimal control was presented by Werbos in 1987 [9].”
Value-gradient learning (2012, methodology): “These algorithms include the original VGL algorithms by Werbos (1992b), which are Dual Heuristic Programming (DHP) and Globalized Dual Heuristic Programming (GDHP); and also the new algorithm of this thesis, VGL(λ).”