Asynchronous gossip algorithms for stochastic optimization

By Sundhar Srinivasan Ram, Angelia Nedić, Venugopal V. Veeravalli. 2009

148 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Public Health, Environmental and Occupational Health

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Public Health, Environmental and Occupational Health

Citations in the corpus

Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent (2017, methodology): “Although decentralized algorithms have been studied as consensus optimization in the control community and used for preserving data privacy [Ram et al., 2009a, Yan et al., 2013, Yuan et al., 2016], for the application scenario where only the decentralized network is available, it is still an open…”
Distributed Alternating Direction Method of Multipliers (2012, background): “The computation at each step can be very inexpensive and lead to distributed implementations [4], [9], [8], [11], [13], [14].”
How to scale distributed deep learning? (2016, methodology): “Asynchronous gossiping SGD was introduced in [10] for the general case of a sparse communication graph between nodes (e.”
Demystifying Parallel and Distributed Deep Learning (2018, neutral): “ Analysis 1:23 Category Method Model Consistency Synchronization Synchronous[45,112,173,210,224,246] Stale-Synchronous[88,98,120,156,261] Asynchronous[55,56,127,182,190,205,260] Nondeterministic Comm.[54,121,201] Parameter Distribution and Communication Centralization Parameter Server (PS)[52,112,12”
Fast Distributed Gradient Methods (2011, background): “The existing literature on distributed optimization studies basically two types of algorithms with inherent tradeoffs: 1) The first type correspond to distributed gradient-like methods, namely, primal (sub) gradient [9], [10], [11], [12], [13], [14], [15], dual averaging [16], [17], [18], and primal”
On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers (2013, background): “Moreover, neither of the works [6] and [10] provides a rate of convergence analysis of the asynchronous algorithm.”
Gossip Algorithms for Distributed Signal Processing (2010, background): “Just as the average can be viewed as the minimizer of a quadratic cost function, researchers are studying what other classes of functions can be optimized within the gossip framework [127].”
Distributed Asynchronous Constrained Stochastic Optimization (2011, background): “We are interested in the case when the communication links are noisy and; hence, node has access to a noise corrupted value of its neighbor’s local estimate.”
On stochastic gradient and subgradient methods with adaptive steplength sequences (2011, background): “Distributed consensus-based stochastic subgradient methods for minimizing a convex objective over a network have been recently developed and studied in [29], [30], [31].”
Asynchronous Broadcast-Based Convex Optimization Over a Network (2011, methodology): “Date of publication September 27, 2010; date of current version June 08, 2011.”
Distributed Algorithms for Aggregative Games on Graphs (2016, background): “This idea has attracted a lot of attention recently in an eﬀort to extend the algorithm of [57] to more general and broader range of problems [35, 46, 45, 19, 18, 36, 54, 33, 26, 58, 48, 59, 5].”
Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method (2013, unknown): “One direction is to extend the algorithm to asynchronous network models such as those considered in [49], [50].”
Privacy preservation in federated learning: An insightful survey from the GDPR perspective (2020, methodology): “centralised server to coordinate the parameter update; instead, each node both locally computes gradients and performs parameter aggregation by communicating with other nodes using a Gossip algorithm [28]–[30]. The ring architecture requires an efﬁcient asynchronous updates strategy among compute no”
Asynchronous Gossip-Based Random Projection Algorithms Over Networks (2013, methodology): “Asynchronous al gorithms based on a gossip scheme have been proposed and analyzed for a scalar objective funct ion and a diminishing stepsize [26], and a vector objective function and a constant stepsize [24] .”
Asynchronous Incremental Stochastic Dual Descent Algorithm for Network Resource Allocation (2017, background): “This approach is followed in [18]–[20], all of which consider various consensus-based distributed subgradient algorithms.”
Private Weighted Random Walk Stochastic Gradient Descent (2020, methodology): “For our decentralized setting, in which data sampling is not independent and is constrained by the graph topology, random walk SGD algorithms can approach the same rate of convergence for convex global loss function [1], [2], [4], [5], [37].”