Applications of advances in nonlinear sensitivity analysis

By Paul J. Werbos. 1982

Subfields: Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Numerical Methods & Scientific Computing

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, methodology): “The backpropagation algorithm [39, 50, 103] can be used to train a large M in form of a neural network (NN).”
Recurrent Highway Networks (2016, background): “However, certain internal function mappings in modern RNNs composed of units grouped in *Equal contribution 1ETH Zürich, Switzerland 2The Swiss AI Lab IDSIA (USI-SUPSI) & NNAISENSE, Switzerland.”
Deep learning in neural networks: An overview (2014, methodology): “To my knowledge, the first NN-specific application of efficient BP as above was described in 1981 (Werbos, 1981, 2006).”
World Models (2018, methodology): “The backpropagation algorithm (Lin-nainmaa, 1970; Kelley, 1960; Werbos, 1982) can be used to train large neural networks efﬁciently.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Typical applications include BP in FNNs [297], or BP through time (BPTT) and similar methods in RNNs, e.”
One Big Net For Everything (2018, background): “In particular, standard gradient descent through backpropagation in discrete graphs of nodes with differentiable activation functions [38, 98][76, Sec.”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “Backpropagation of error (Werbos, 1974, 1982; Rumelhart et al., 1986) enabled neural networks to be trained eﬃciently, by providing an eﬃcient means to compute the gradient with respect to the weights of a multi-layer network.”
Reinforcement Learning: An Introduction (1998, background): “Werbos (1977) suggested an approach to approximating DP called “heuristic dynamic programming” that emphasizes gradient-descent methods for continuous-state problems (Werbos, 1982, 1987, 1988, 1989, 1992).”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, methodology): “To achieve this, the gradient of the loss is computed with respect to each learnable parameter by applying the chain rule from the final layer back to each weight [152], [153], [154].”
Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (1992, methodology): “Globalized DHP (GDHP), which adapts a Critic network whose output is an approximation of J(R(t», but adapts it so as to minimize errors in the implied derivatives of J (as well as J itself, with some weighting) [35,36,37].”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, methodology): “Replay-based training on previous behaviors and commands compatible with observed time horizons and costs: For all pairs {(k, j); 1 ≤ k ≤ j ≤ t}: train C through gradient descent-based backpropagation [28, 24, 69][52, Sec.”
Deep Learning and Its Applications in Biomedicine (2018, methodology): “Thanks to the improvement in computer capabilities and methodologies [18], ANNs with efﬁcient backpropagation (BP) facilitated studies on pattern recognition [19–23].”