The numerical solution of variational problems

By Stuart E. Dreyfus. 1962

In two earlier papers [l, 21 we have derived various conditions that must be satisfied by a curve if it is to be a solution of a variational problem. In [l] we considered mainly the classical problem of Lagrange. In [2] we studied the problem of Mayer and investigated the conditions implied by the introduction of inequality constraints on either the shape of the solution curve (decision variable constraints) or the region in which it could lie (state variable constraints). In this paper we shall discuss the numerical solution of variational problems of both the conventional and inequality-constrained types. We shall outline briefly what, until recently, has been the usual approach. Then we shall consider a gradient technique that has proved very successful in practice. This approach is known to a few practitioners of the numerical art such as Kelley [3] and Bryson [4]. Our derivation, using essentially the same concepts and techniques of dynamic programming that we employed in ref. 1 and 2, is both new and simple. We conclude this paper with a discussion of the numerical solution of a variant of the classical brachistochrone problem where a state variable inequality constraint has been introduced but for which the analytic solution is still known. We shall assume that the reader is familiar with such dynamic programming terms as “state variable,” “decision variable,” “optimal return function,” etc., that were introduced in our two previous papers and also with the different Lagrange multiplier relationships that hold on and off the boundary, as derived in [2].

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “A simplified derivation uses the chain rule only (Dreyfus, 1962).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “To maximize differentiable objective functions of SL and UL, NN researchers almost invariably use backpropagation (BP) [125, 30, 52] in discrete graphs of nodes with differentiable activation functions [151, 265][245, Sec.”
Artificial intelligence to deep learning: machine intelligence approach for drug discovery (2021, background): “Henry J. Kelley developed the continuous backpropagation model in 1960, and a simpler version based only on-chain rule was developed by Stuart Dreyfus in 1962 [22, 23].”
Plant Disease Detection and Classification by Deep Learning (2019, methodology): “During the ﬁrst phase, several developments like backpropagation [2,3], chain rule [4], Neocognitron [5], hand written text recognition (LeNET architecture) [6], and resolving the training problem [7,8] were observed (as shown in Figure 1).”
My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013 (2013, background): “Gradient descent [42] in such systems can be performed [13, 50, 12] by iterating the ancient chain rule [57, 58] in dynamic programming style [9] (compare simpliﬁed derivation using chain rule only [27]).”
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017, methodology): “In particular, S. E. Dreyfus (1962) derived a simplified version of the algorithm that used the chain rule alone.”
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics (2019, methodology): “Backpropagation dates back to the works of Dreyfus (1962), Linnainmaa (1976) and Rumelhart, Hinton, and Williams (1986), while its extension to RNNs termed Backpropagation through time (BPTT) was presented in Werbos (1988, 1990).”
Model-Based Planning in Discrete Action Spaces (2017, background): “The idea of planning actions by backpropagating along a policy trajectory has existed since the 1960’s [19, 7].”