Linear Stochastic Approximation: How Far Does Constant Step-Size and Iterate Averaging Go?

147 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values (2020, methodology): “Note it is possible to conduct a ﬁnite sample analysis without introducing projection using arguments from Lakshminarayanan & Szepesvari (2018), which we leave for future work.”
Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning (2019, methodology): “A number of temporal difference algorithms for reinforcement learning, including TD(0), TD(λ) and GTD; and stochastic gradient descent algorithm for linear-square estimation can be written in this form (see the detailed discussion in Lakshminarayanan and Szepesvari (2018)). We are interested in esti”
Recent advances in reinforcement learning in finance (2021, background): “…converges at a rate of ̃′( 1 𝜀1∕𝜎 ) with decaying learning rate 𝛽𝑛 = 1 (1+𝑛)𝜎 for some 𝜎 ∈ (0, 1) where i.i.d. samples are drawn from a stationary distribution; Lakshminarayanan and Szepesvari (2018) provided an improved ̃′( 1 𝜀 ) bound for iterate-averaged TD(0) with constant step-size;…”
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation (2018, methodology): “Instead, many papers have studied such algorithms under the simpler i.i.d noise model mentioned earlier [Sutton et al., 2009b,a, Liu et al., 2015, Touati et al., 2018, Dalal et al., 2018b, Lakshminarayanan and Szepesv´ari, 2018].”