Asynchronous Gossip Algorithm for Stochastic Optimization: Constant Stepsize Analysis*

By Sundhar Srinivasan Ram, Angelia Nedić, Venugopal V. Veeravalli. 2010

54 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Public Health, Environmental and Occupational Health, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Public Health, Environmental and Occupational Health

In Optimization & Training

Citations in the corpus

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (2020, methodology): “Whilst no explicit values of p were given for this approach, for the simpler instance of • pairwise randomized gossip (Boyd et al., 2006; Ram et al., 2010; Lee & Nedić, 2015; Loizou & Richtárik, 2019) we have p = Θ ( 1 n2 ) , thus by sampling a linear number of (independent) edges—not necessarily…”
Communication Efficient Decentralized Training with Multiple Local Updates (2019, methodology): “Decentralized (stochastic) algorithms used to tackle the failure of being centralized as a compromise and used to be studied as consensus optimization in the control community [28, 50, 36].”
D2: Decentralized Training over Decentralized Data (2018, methodology): “The D-PSGD algorithm [Nedic and Ozdaglar, 2009, Ram et al., 2010a,b] requires each worker to compute a stochastic gradient and exchange its local model with neighbors.”
Asynchronous Decentralized Parallel Stochastic Gradient Descent (2017, background): “Ram et al. [2010] proposed an asynchronous subgradient variations of the decentralized stochastic optimization algorithm for convex problems.”
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent (2017, background): “…and Nedic, 2011, Sundhar Ram et al., 2010] or implicitly indicate that decentralized algorithms were much worse than centralized algorithms in terms of computational complexity and total communication complexity [Aybat et al., 2015, Lan et al., 2017, Ram et al., 2010, Zhang and Kwok, 2014].”
Balancing Communication and Computation in Distributed Optimization (2017, background): “A number of later contributions [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30] extended DGD to other settings, including stochastic networks, constrained problems, and noisy environments.”
Communication-Efficient Local Decentralized SGD Methods (2019, background): “They were studied as consensus optimization in the control community [25, 45, 31].”
Asynchronous Gossip-Based Random Projection Algorithms Over Networks (2013, methodology): “We compare the GRP algorithm with the Distributed Gossip Algorithm (DGA) in [15] that utilizes the complete projection on X at every iteration.”
Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes (2015, methodology): “This is analogous to using different stepsizes in the average sense especially when it comes to random networks [21], [22].”