Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences

By Paul J. Werbos. 1974

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Semi-supervised Sequence Learning (2015, background): “Recurrent neural networks (RNNs) are powerful tools for mod eling sequential data, yet training them by back-propagation through time [36, 26] can be difficu lt [8].”
Multi-column deep neural networks for image classification (2012, background): “(2) It was shown [14] that such multi-layered DNN are hard to train by standard gradient descent [36, 18, 28], the method of choice from a mathematical/algorithmic point of view.”
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition (2010, background): “Are all these complexiﬁcations of plain MLPs really necessary?”
Wireless Network Intelligence at the Edge (2018, methodology): “Training an MLP is performed using a gradient descent optimization algorithm, called the backpropagation method [73].”
Deep learning in neural networks: An overview (2014, neutral): “Compare some preliminary, NN-specific discussion (Werbos, 1974, Section 5.5.1), a method for multilayer threshold NNs (Bobrowski, 1978), and a computer program for automatically deriving and implementing BP for given differentiable systems (Speelpenning, 1980).”
A Brief Survey of Deep Reinforcement Learning (2017, neutral): “The workhorse of DRL, however, remains backpropagation [162, 111].”
Optimal Ordered Problem Solver (2002, methodology): “Other primitives may include various traditional AI path planners (Russell and Norvig, 1994), artiﬁcial neural networks (Werbos, 1974, Rumelhart et al., 1986, Bishop, 1995) or support vector machines (Vapnik, 1992) for classifying sensory data written into temporary internal storage, as well as…”
DeepTox: Toxicity Prediction using Deep Learning (2016, methodology): “For neural networks , gradient descent can be applied with high computational efficiency by using the backpropagation algorithm ( Werbos , 1974 ; Rumelhart et al .”
Distributed optimization of deeply nested systems (2012, methodology): “, this is challenging because nesting (i.e., function composition) produces inherently nonconvex functions. Joint training is usually done with the backpropagation algorithm (Rumelhart et al., 1986a; Werbos, 1974), which recursively computes the gradient with respect to each parameter using the chai”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “Backpropagation of error (Werbos, 1974, 1982; Rumelhart et al., 1986) enabled neural networks to be trained eﬃciently, by providing an eﬃcient means to compute the gradient with respect to the weights of a multi-layer network.”
A Competitive Modular Connectionist Architecture (1990, methodology): “During training, the weights of the expert and gating networks are adjusted simultaneously using the backpropagation algorithm (Ie Cun, 1985; Parker, 1985; Rumelhart, Hinton, and Williams, 1986; Werbos, 1974) so as to maximize the function”
When Neurons Fail (2017, methodology): “Neural networks received a regain of interest in the late 1970s with the backpropagation algorithm [4] that overcame learning issues.”
Opening the black box - data driven visualization of neural networks (2005, background): “Our designs and (cid:2)ndings have helped us develop better intelligent visualization systems, and should also help others gain both understanding and con(cid:2)dence in using ANNs.”
Time series prediction by using a connectionist network with internal delay lines (1993, unknown): “Previous work includes that of Werboss7, 8], Lapedes 9], and Weigend et al 10] to cite just a few.”
Artificial neural networks (2001, unknown): “The capabilities of neural networks were expanded from linear to nonlinear domains in 1974 by Werbos, 4 with the intention of developing a method as general as linear regression, but with nonlinear capabilities.”