On the Convergence of Decentralized Gradient Descent

By Kun Yuan, Qing Ling, Wotao Yin. 2013

766 citations recorded by the source corpus.

Consider the consensus problem of minimizing $f(x)=\sum_{i=1}^n f_i(x)$, where $x\in{\mathbb{R}}^p$ and each $f_i$ is only known to the individual agent $i$ in a connected network of $n$ agents. To solve this problem and obtain the solution, all the agents collaborate with their neighbors through information exchange. This type of decentralized computation does not need a fusion center, offers better network load balance, and improves data privacy. This paper studies the decentralized gradient descent method [A. Nedic and A. Ozdaglar, IEEE Trans. Automat. Control, 54 (2009), pp. 48--61], in which each agent $i$ updates its local variable $x_{(i)}\in{\mathbb{R}}^n$ by combining the average of its neighbors' with a local negative-gradient step $-\alpha \nabla f_i(x_{(i)})$. The method is described by the iteration $ x_{(i)}(k+1) \gets \sum_{j=1}^n w_{ij} x_{(j)}(k) - \alpha \nabla f_i(x_{(i)}(k)),\ \text{for each agent}~i, $ where $w_{ij}$ is nonzero only if $i$ and $j$ are neighbors or $i=j$ and the matrix...

Subfields: Computer Networks and Communications, Optimization & Training, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

  1. Angelia Nedić
  2. Ali H. Sayed
  3. Wotao Yin
  4. Wei Shi
  5. Aryan Mokhtari
  6. Alexander Olshevsky
  7. Qing Ling
  8. Kun Yuan

In Optimization & Training

  1. Aryan Mokhtari
  2. Peter Richtárik
  3. Nicolas Loizou
  4. Martin Jaggi
  5. Sebastian U. Stich
  6. Wotao Yin
  7. Angelia Nedić
  8. Ji Liu

In Federated & Distributed Learning

  1. Jianyu Wang
  2. Gauri Joshi
  3. Aryan Mokhtari
  4. Martin Jaggi
  5. Ji Liu
  6. Shusen Wang
  7. Zhihua Zhang
  8. Xiang Li

Citations in the corpus