wchastic. approximation methods for constrained and unconstrained systems

By Harold J. Kushner, D. Clark. 1978

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

On Convergence of Emphatic Temporal-Difference Learning (2015, background): “The four remaining conditions to be introduced are of the same type and relate to the asymptotic rate of change conditions introduced by (Kushner and Clark, 1978).”
A review of simulation optimization techniques (1998, background): “Similar ideas have been proposed by Kushner and Clark ( ) and Ermoliev (1983).”
Optimal Stochastic Search and Adaptive Momentum (1993, background): “Kushner and Clark (1978) define continous time functions that interpolate the discrete time process in order to establish an equivalence between the asymptotic behavior of the discrete time stochastic process, and solutions of an associated deterministic differential equation.”
Weak Convergence Properties of Constrained Emphatic Temporal-difference Learning with Constant and Slowly Diminishing Stepsize (2015, methodology): “The first group of results are derived by combining key properties of ETD(λ) with powerful convergence theorems from the weak convergence methods in stochastic approximation theory [17, 18, 19].”
A Concentration Bound for Stochastic Approximation via Alekseev’s Formula (2015, result): “ich by [7, Corollary 5.29, p96], implies that X n I{A(n),xn ∈ B} < ∞ a.s. In particular, this implies that xn → x∗ a.s. on the set {xn ∈ B i.o.}. Thus we recover the celebrated Kushner-Clark lemma [15] under the weaker hypothesis an → 0 replacing the usual condition P n a 2 n < ∞. The rest of ”
Settable Systems: An Extension of Pearl's Causal Model with Optimization, Equilibrium, and Learning (2009, methodology): “A general learning algorithm introduced by Kushner and Clark (1978) (KC) has the form”
Pattern-recognizing stochastic learning automata (1985, methodology): “Another proof provided by Lakshmivarahan [1] which can also be extended to the associative case is based on Kushner's method of asymptotic analysis as described in [46].”
A unified framework for stochastic optimization (2019, methodology): “A separate line of research focused on constrained probems under the umbrella of “stochastic quasi-gradient” methods, ith seminal contributions from Ermoliev (1983) , Shor (1979) , flug (1988b) , Kushner and Clark (1978) , Shapiro and Wardi (1996) , nd Kushner and Yin (2003) .”
Q-learning (1992, background): “Unfortunately, the theorem does not extend trivially to this case, and alternative proof methods such as those in Kushner and Clark (1978) may be required.”
Monte Carlo sampling-based methods for stochastic optimization (2014, background): “A different line of work involves the use of ordinary differential equations to analyze the behavior of SA algorithms; such an approach was introduced by Kushner and Clark [139] and elaborated further in Kushner and Yin [140], see also Borkar and Meyn [31].”
Two-Timescale Networks for Nonlinear Value Function Approximation (2019, methodology): “We employ here the ODE based analysis as proposed in (Borkar, 2008; Kushner and Clark, 2012).”
Stochastic Consensus Seeking With Noisy and Directed Inter-Agent Communication: Fixed and Randomly Varying Topologies (2010, background): “Equality (12) is usually called the Kushner-Clark condition [21], [33] along that noise sample path.”
Evaluation of Optimization Methods for Nonrigid Medical Image Registration Using Mutual Information and B-Splines (2007, background): “…descent, see (8), with the distinction that the derivative of the cost function, , is replaced by an approximation , resulting in the following scheme: (20) Convergence to the solution can only be guaranteed [11] if the bias of the approximation error goes to zero (21) where denotes expectation.”