Decentralized consensus optimization with asynchrony and delays

By Tianyu Wu, Kun Yuan, Qing Ling, Wotao Yin, Ali H. Sayed. 2016

134 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Mathematical Optimization

Citations in the corpus

Central Server Free Federated Learning over Single-sided Trust Social Networks (2019, background): “Wu et al. (2017); Shen et al. (2018) shows that the decentralized SGD would converge with a comparable convergence rate to the centralized algorithm with less communication to make large-scale model training feasible.”
Robust and Communication-Efficient Collaborative Learning (2019, background): “Another line of work proposes asynchronous decentralized SGD, where the workers update their models based on the last iterates received by their neighbors (Recht et al., 2011; Lian et al., 2017b; Lan and Zhou, 2018; Peng et al., 2016; Wu et al., 2017; Dutta et al., 2018).”
Qsparse-Local-SGD: Distributed SGD With Quantization, Sparsification, and Local Computations (2019, background): “This is diﬀerent from asynchronous algorithms studied for stragglers [WYL + 18, RRWN11], where only one gradient step is taken but occurs at diﬀerent times due to delays.”
Decentralized Online Learning: Take Benefits from Others’ Data without Sharing Your Own to Track Global Trend (2019, background): “It is assumed to be a doubly stochastic matrix [34, 39, 40] but not necessarily symmetric.”
COLA: Decentralized Linear Learning (2018, methodology): “For the decentralized, asynchronous consensus optimization, [Wu et al., 2018] extends the existing PG-EXTRA and proves convergence of the algorithm.”
Asynchronous Decentralized Parallel Stochastic Gradient Descent (2017, methodology): “1 Decentralized algorithms are mainly studied as consensus optimization problems by the control community [Ram et al., 2010, Yan et al., 2013, Yuan et al., 2016] for the application scenario where only a decentralized network is available.”
Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communication (2018, methodology): “Inspired by (Wu et al., 2016), we introduce the positive definite matrix”
Supervised Learning Under Distributed Features (2018, background): “Fortunately, problems involving delays in gradient calculations are well-studied in the distributed machine learning and signal processing literature [57]–[59].”
A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates (2017, methodology): “EXTRA is equivalent to the Condat-Vu primal-dual algorithm [47], [55], and it can be further explained as a forward-backward”
Communication-Censored ADMM for Decentralized Consensus Optimization (2019, background): “This brief survey is also far from complete; for example, it does not include asynchronous algorithms that are important to heterogeneous networks [30].”
Optimization over time-varying directed graphs with row and column-stochastic matrices (2018, methodology): “Furthermore, asynchronous multi-agent optimization is considered in [46], where the authors adapt the curvature estimation technique of the Broyden-Fletcher-Goldfarb-Shanno quasi-Newton optimization method [47]–[49] for use in asynchronous distributed settings over undirected graphs [50].”
Push–Pull Gradient Methods for Distributed Optimization in Networks (2018, background): “Paper [30] extends the work in [2] to handle asynchrony and delays.”
Decentralized Consensus Optimization With Asynchrony and Delays (2016, background): “This paper was presented at the 2016 Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, November 6–9, 2016 [1].”
Asynchronous Decentralized Online Learning (2021, methodology): “Similar to the simulation method in [25], the computation time of learner i follows the exponential distribution Exp(μi) with mean value μi = |μ̄|+ 2, where μ̄ is sampled from the standard normal distribution Normal(0, 1).”