Better Communication Complexity for Local SGD

30 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Advances and Open Problems in Federated Learning (2019, methodology): “ingle device (ignoring the updates of the other devices) achieves the optimal ‘optimization’ term (H=(TK)2). The convergence analysis of local-update SGD methods is an active current area of research [370, 271, 428, 399, 334, 318, 233]. The ﬁrst convergence results for local-update SGD methods were ”
Is Local SGD Better than Minibatch SGD? (2020, methodology): “However, consider the case of arbitrary H , B and σ 2 and suppose Khaled et al. (2019)’s guarantee is less than σB √ KR , in which case Consequently, (22) is either greater than σB √ KR or greater than HB 2 R .”
Tighter Theory for Local SGD on Identical and Heterogeneous Data (2019, background): “Our earlier workshop paper (Khaled et al., 2019a) Assump-tion 2, however, typically does not hold for ﬁnite-sum problems where g p x, z q is a gradient of the one functions in the ﬁnite-sum.”
Minibatch vs Local SGD for Heterogeneous Distributed Learning (2020, methodology): “[11] and Koloskova et al.”
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning (2021, background): “Since then, this work has sparked many follow-ups that focus on FL with i.i.d. datasets and full worker participation (also known as LocalSGD (Stich, 2018; Yu et al., 2019b; Wang & Joshi, 2018; Stich & Karimireddy, 2019; Lin et al., 2018; Khaled et al., 2019a; Zhou & Cong, 2017)).”