Quantitative Central Limit Theorems for Discrete Stochastic Processes

By Xiang Cheng, Peter L. Bartlett, Michael I. Jordan. 2019

5 citations recorded by the source corpus.

In this paper, we establish a generalization of the classical Central Limit Theorem for a family of stochastic processes that includes stochastic gradient descent and related gradient-based algorithms. Under certain regularity assumptions, we show that the iterates of these stochastic processes converge to an invariant distribution at a rate of $O\lrp{1/\sqrt{k}}$ where $k$ is the number of steps; this rate is provably tight.

Subfields: Artificial Intelligence, Statistics and Probability