Communication-Efficient Local Decentralized SGD Methods

By Xiang Li, Wenhao Yang, Shusen Wang, Zhihua Zhang. 2019

76 citations recorded by the source corpus.

Subfields: Optimization & Training, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Sebastian U. Stich
2. Martin Jaggi
3. Peter Richtárik
4. Anastasia Koloskova
5. Nicolas Loizou
6. Shiva Prasad Kasiviswanathan
7. Dmitrii Avdiukhin
8. Sadra Boreiri

In Federated & Distributed Learning

1. Martin Jaggi
2. Anastasia Koloskova
3. Dmitrii Avdiukhin
4. Shiva Prasad Kasiviswanathan
5. Nicolas Loizou
6. Sadra Boreiri
7. Sebastian U. Stich
8. Othmane Marfoq

Citations in the corpus

• A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (2020, background): “…SGD (Wang & Joshi, 2018) which considers only the IID data case ( f i = f j ) with local updates and a fixed mixing matrix W , and the recently proposed • periodic decentralized SGD (Li et al., 2019) that allows for multiple local update and multiple mixing steps (for fixed W ) in a periodic manner.”
• Throughput-Optimal Topology Design for Cross-Silo Federated Learning (2020, methodology): “In order to solve Problem (1) in an FL scenario, silos do not share the local datasets, but periodically transmit model updates, and different distributed algorithms have been proposed [57, 72, 58, 45, 104, 52, 103].”
• Federated Learning under Arbitrary Communication Patterns (2021, background): “…on synchronous local SGD with non-identical data distribution across clients wherein all clients communicate their local parameter to the server every fxed Δ rounds (Yu et al., 2019b; Haddadpour & Mahdavi, 2019; Khaled et al., 2019b;a; Wang et al., 2019c; Basu et al., 2019; Li et al., 2019b).”