A Delay-tolerant Proximal-Gradient Algorithm for Distributed Learning

By Konstantin Mishchenko, Franck Iutzeler, Jérôme Malick, Massih-Reza Amini. 2018

40 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

First Analysis of Local GD on Heterogeneous Data (2019, methodology): “[11] proposed an asynchronous local method that converges to the exact solution without decreasing stepsizes, but its benefit from increasing H is limited by constant factors.”
Distributed Learning with Sparse Communications by Identification (2018, methodology): “ construction of our algorithm, based on the average of the parameter rather than only gradients, allows us to have simple steps in the proof. We just need to introduce judicious variables, following [22]. First, from the unique solution x?of (1), let us dene for any slave ithe point x? i := x? rf i”
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it (2019, methodology): “We believe that it possible to extend the works [24, 23] to design a method with proximable regularizer (for `1 penalty) that would communicate little both sides.”
Asynchronous Distributed Learning with Sparse Communications and Identification (2018, background): “This originality is inspired from [22, 21]: though it may appear conservative, it actually performs well in practice due to the stability of the produced iterations; the intuition being that combining delayed points is more stable than using a combination of delayed directions; see the numerical…”
99% of Parallel Optimization is Inevitably a Waste of Time (2019, background): “We believe that it possible to extend the works (Mishchenko et al., 2018b;a) to design a method with proximable regularizer (for `1 penalty) that would communicate little both sides.”
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks (2019, methodology): “papers (see Shamir and Srebro (2014) for the finite sum case, and Olshevsky et al. (2019) where authors provided an asymptotic network independent sublinear rate2), this case has not been the focus of study.”