99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it

9 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Gradient Descent with Compressed Iterates (2019, methodology): “Further, while there is ample work on non-local methods which communicate compressed gradients [1, 3, 11, 34, 35], including methods which perform variance-reduction to remove the variance introduced by compression [10, 23, 24], to the best of our knowledge there is little work on methods performing”
Stochastic Distributed Learning with Gradient Quantization and Variance Reduction (2019, methodology): “O C ] 1 0 A pr 2 01 9 compress it, but to update a subset of elements of the iterate x using coordinate descent type updates (Richtárik & Takáč, 2016; Fercoq et al., 2014; Mishchenko et al., 2019).”
One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methods (2019, methodology): “In particular, we obtain the following methods in special cases, or their generalizations, always recovering the best-known convergence guarantees or improving upon them: SAGA [3, 27, 4], JacSketch [9], LSVRG [12, 15], SEGA [10], and ISEGA [21] (see Table 1, in which we list 17 special cases).”