On Asymptotic Normality in Stochastic Approximation

By V. Fabian. 1968

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Communication trade-offs for synchronized distributed SGD with large step size (2019, background): “This problem has received attention from various communities over the last fifty years in optimization, stochastic approximation, and machine learning (Polyak and Juditsky, 1992; Ruppert, 1988; Fabian, 1968; Nesterov and Vial, 2008; Nemirovski et al., 2009; Shalev-Shwartz et al., 2009; Zhang, 2004).”
A one-measurement form of simultaneous perturbation stochastic approximation (1997, methodology): “(1992), which were based on an asymptotic distribution theory in Fabian (1968) (the author is unaware of any formal distribution theory for the finite-sample case).”
Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning (2011, result): “Moreover, our results are consistent with the asymptotic results from [6].”
Optimization (2005, unknown): “Fabian, V. (1968). On asymptotic normality in stochastic approximation.”
Evaluation of Optimization Methods for Nonrigid Medical Image Registration Using Mutual Information and B-Splines (2007, background): “More details can be found in [11], [17], [35], and [36].”
Curiously Fast Convergence of some Stochastic Gradient Descent Algorithms (2009, background): “Various theoretical works [2, 3, 6] indicate that no choice of η t can lead to faster convergence rates than t .”
Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression (2013, methodology): “Note that the often advocated step size, that is, of the form C/n where C is larger than 1 /µ , leads, without averaging to a convergence rate of O (1 /µ 2 n ) (Fabian, 1968; Bach and Moulines, 2011), hence with a worse dependence on µ .”
Averaged Least-Mean-Squares: Bias-Variance Trade-offs and Optimal Sampling Distributions (2015, methodology): “For decaying step-sizes and general loss functions, this was partially considered by Fabian (1968), who provides the limiting distribution of iterates (from which the generalization performance can be derived), but only without averaging.”
Design Issues for Generalized Linear Models: A Review (2006, background): “Fabian (1968) proved the asymptotic normality of xn under additional assumptions.”
Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent (2011, background): “We note that the asymptotic normality of θ t (Fabian, 1968) suggests that assumption (13) is reasonable.”
Statistical inference for model parameters in stochastic gradient descent (2016, background): “Back in 1960s, [10] studied the distribution of SGD iterates.”