Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization

By Xianghui Mao, Kun Yuan, Yubin Hu, Yuantao Gu, Ali H. Sayed, Wotao Yin. 2018

72 citations recorded by the source corpus.

This paper introduces a new algorithm for consensus optimization in a multi-agent network, where all agents collaboratively find a minimizer for the sum of their private functions. All decentralized algorithms rely on communications between adjacent nodes. One class of algorithms use communications between some or all pairs of adjacent agents at each iteration. Another class of algorithms uses a random walk incremental strategy, which sequentially activates a succession of agents. Existing incremental algorithms require diminishing step sizes to converge to the solution, and their convergence is slow. In this work, we propose a random walk algorithm that uses a fixed step size and converges faster to the solution than the existing random walk incremental algorithms. Our algorithm uses only one link to communicate the latest information from an agent to another. Since this style of communication mimics a man walking in a network, we call our algorithm Walkman. We establish convergence for convex and nonconvex objectives. For decentralized least squares, we derive a linear rate of convergence and obtain a better communication complexity than those of other decentralized algorithms. Numerical experiments verify our analysis results.

Subfields: Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Qing Ling
2. Gang Wu
3. Aurélien Bellet
4. Edwige Cyffers
5. Salim el Rouayheb
6. Ghadir Ayache
7. Zhi Tian
8. Wei Xu

Citations in the corpus

• Privacy Amplification by Decentralization (2020, methodology): “…a way to perform (non-private) decentralized estimation and optimization with less communication and computation overhead than algorithms that require all users to communicate with their neighbors at each step (Ram et al., 2009; Johansson et al., 2009; Mao et al., 2020; Ayache and Rouayheb, 2020).”
• Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges (2022, methodology): “These include Random Walk ADMM (WADMM) [134], Parallel Random Walk ADMM (PW-ADMM) [123] and Walk Proximal Gradient (WPG) [135] which are commonly used in increment-based approaches.”
• Communication-Censored ADMM for Decentralized Consensus Optimization (2019, methodology): “Other efforts to reduce the communication cost include the random-walk ADMM that randomly activates a succession of nodes and incrementally updates the optimization variable [34], and the block-iterative method that updates and communicates only a block of a highdimensional optimization variable at ”
• Private Weighted Random Walk Stochastic Gradient Descent (2020, background): “In [9], the authors studied the convergence analysis of random walk learning algorithm for the alternating direction method of multipliers (ADMM).”