Communication-Efficient Stochastic Gradient Descent, with Applications to Neural Networks

By Dan Alistarh, Demjan Grubic, Jerry Liu, Ryota Tomioka, Milan Vojnovic. 2017

7 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Felix X. Yu
2. Ananda Theertha Suresh
3. Jakub Konecný
4. Sanjiv Kumar
5. H. Brendan McMahan
6. Peter Kairouz
7. Naman Agarwal
8. Takuya Akiba

Citations in the corpus

• cpSGD: Communication-efficient and differentially-private distributed SGD (2018, background): “To address this issue, several recent works have focused on reducing the communication cost of distributed learning algorithms via gradient quantization and sparsification [33, 17, 34, 20, 21, 4, 35].”
• Variance-based Gradient Compression for Efficient Distributed Deep Learning (2018, methodology): “We do not adopt stochastic rounding like Alistarh et al. (2017) nor accumulate rounding error g i − g i for the next batch because this simple rounding does not harm accuracy empirically.”