A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications

By H. Robbins, David O. Siegmund. 1985

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variable Metric Stochastic Approximation Theory (2009, background): “1 (Robbins and Siegmund, 1971) Let (Ω,F , P ) be a probability space and F1 ⊆ F2 ⊆ .”
Mean-normalized stochastic gradient for large-scale deep learning (2014, background): “[18] H. E. Robbins and D. O. Siegmund, “A convergence theorem for non negative almost supermartingales and some applications,” in Proc.”
ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems and GANs (2019, unknown): “Lemma 2 (Robbins–Siegmund [57]) .”
Extragradient Method with Variance Reduction for Stochastic Variational Inequalities (2017, methodology): “As in other stochastic approximation methods, a fundamental tool to be used is the following convergence theorem of Robbins and Sieg-mund (see [36]).”
The dropout learning algorithm (2014, background): “stochastic gradient descent in standard neural networks [38, 13, 14].”
Understanding Dropout (2013, background): “As such, and with the appropriately decreasing learning rates, it is almost surely convergent like other forms of stochastic gradient descent [11, 4, 5].”
Stochastic Learning on Imbalanced Data: Determinantal Point Processes for Mini-batch Diversification (2017, unknown): “Stochastic gradient descent (SGD) is one of the most important algorithms for scalable machine learning [7, 35].”
Stochastic Gradient Descent Tricks (2012, background): “The Robbins-Siegmund theorem [16] provides the means to establish almost sure convergence under surprisingly mild conditions [3], including cases where the loss function is non smooth.”
The Statistics of Streaming Sparse Regression (2014, background): “Note that, while Regret(u) appears to measure loss on a training set, it is actually more closely related to generalization error, since wt is chosen before observing ft, and thus there is no opportunity for wt to be overfit to the function ft.”
ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates (2015, background): “We need the following lemma on nonnegative almost supermartingales [50].”
Towards closing the gap between the theory and practice of SVRG (2019, methodology): “The expected smoothness constant has been instrumental in providing a tight mini-batch analysis for SGD [8], SAGA [7] and now SVRG.”
A Stochastic Quasi-Newton Method for Online Convex Optimization (2007, unknown): “We are developing an alternative path to establish the convergence of oBFGS, based on the work of Robbins and Siegmund (1971).”
Martingales in Sequential Analysis and Time Series, 1945-1985 ∗ (2009, methodology): “Chow, Y. S. and Robbins, H. (1965).”