Asynchronous Stochastic Approximations

By Vivek Shripad Borkar. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning with Elastic Averaging SGD (2014, methodology): “…the distinctive feature of EASGD is that it allows the local workers for signiﬁcantly more exploration as opposed to ADMM -like methods, DOWNPOUR and many others Borkar (1998); Nedi´c et al. (2001); Langford et al. (2009); Agarwal & Duchi (2011); Recht et al. (2011); Zinkevich et al. (2010).”
Asynchronous Broadcast-Based Convex Optimization Over a Network (2011, background): “A very different distributed computational model has been proposed in [15] and also discussed in [16]–[18], where the network objective function is available to each agent and the processing is distributed by allowing each agent to update only a part of the decision vector.”
Algorithms for Reinforcement Learning (2010, unknown): “…assuming that the step-size sequence satisfies the Robbins-Monro (RM) conditions, ∞∑ t=0 αt =∞, ∞∑ t=0 α2t < +∞, the sequence (V̂t ∈ RD; t ∈ N) will track the trajectories of the ordinary differential equation (ODE) v̇(t) = c F (v(t)), t ≥ 0, (18) where c = 1/D and v(t) ∈ RD (e.g., Borkar, 1998).”
A Reinforcement Learning Algorithm Based on Policy Iteration for Average Reward: Empirical Results with Yield Management and Convergence Analysis (2004, methodology): “The ODE analysis in Borkar (1998) shows that even under asynchronous conditions, there exists a similar ODE, and convergence is guaranteed under asynchronous conditions as well.”
The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning (2000, background): “One argues exactly as above and in [6] to conclude that the suitably interpolated and rescaled trajectory of the algorithm tracks an appropriate o.”
Learning Algorithms for Markov Decision Processes with Average Cost (2001, methodology): “The rest then follows as in the KushnerClark approach [14] in synchronous case and by Borkar’s theorem [9] in the asynchronous case.”
Asynchronous gossip algorithms for stochastic optimization (2009, background): “A different distributed model has been proposed in [31] and also studied in [2], [5], [32], where the complete objective function information is available to each agent, with the aim of distributing the processing by allowing an agent to update only a part of the decision vector.”
Q-learning and enhanced policy iteration in discounted dynamic programming (2010, background): “[46], Bertsekas and Tsitsiklis [12], Borkar [16], Abounadi et al.”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, background): “The idea of ordinary differential equations (ODEs) for proving convergence under asynchronous conditions was proposed in Borkar (1998), where it was shown that the iterate tracks an ODE, which is much slower than that shown to exist under synchronous conditions (Kushner and Clark, 1978).”
Distributed stochastic optimization for deep learning (thesis) (2016, unknown): “Another possibility is to use the asynchronous stochastic gradient descent methods [8, 29, 63, 2, 46, 16].”
Incremental Stochastic Subgradient Algorithms for Convex Optimization (2008, methodology): “The Markov randomized incremental algorithm is also related to the decentralized computation model in [3, 6, 40] for stochastic optimization problems.”