Parallel Gradient Distribution in Unconstrained Optimization

By Olvi L. Mangasarian. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

First Analysis of Local GD on Heterogeneous Data (2019, methodology): “[9] seems to be the ﬁrst work to propose a local method, but no rate was shown in that work. respectively.”
An asynchronous parallel stochastic coordinate descent algorithm (2013, background): “Similar ideas appeared in [Mangasarian, 1995], with a diﬀerent synchronization step.”
A reliable effective terascale linear learning system (2011, background): “Several authors (Mangasarian, 1995; McDonald et al., 2010; Zinkevich et al., 2010) have studied approaches that first solve the learning problem independently on each machine using the portion of the data stored on that machine, and then average the independent local solutions to obtain the global s”
Better Communication Complexity for Local SGD (2019, methodology): “The history of the methods goes back to the convergence proof in the early work [11], but a tight convergence rate has been missing since then.”
Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties (2014, methodology): “Our discussion there included non-stochastic, cyclic coordinate descent methods [38, 23, 41, 5, 39, 40], synchronous parallel methods that distribute the work of function and gradient evaluation [15, 24, 17, 7, 11, 1, 10, 35], and asynchronous parallel stochastic gradient methods (including the rand”
Federated Learning under Arbitrary Communication Patterns (2021, background): “Such a strategy is commonly referred to as local SGD (Mangasarian, 1995; Zinkevich et al., 2010; Coppola, 2015; Stich, 2018; Zhou & Cong, 2018), but is also known in the literature under various other names (such as parallel SGD ) (Yu et al., 2019b).”
SSVM: A Smooth Support Vector Machine for Classification (2001, background): “2 (iv) of [20] we have that each accumulation point of {x} is stationary.”
A coordinate gradient descent method for nonsmooth separable minimization (2008, background): “This coordinate gradient descent approach may be viewed as a hybrid of gradient-projection and coordinate descent; also see [15,20,31].”