Asynchronous Gradient Push

By Mahmoud Assran, Michael G. Rabbat. 2018

77 citations recorded by the source corpus.

We consider a multiagent framework for distributed optimization where each agent has access to a local smooth strongly convex function, and the collective goal is to achieve consensus on the parameters that minimize the sum of the agents’ local functions. We propose an algorithm wherein each agent operates asynchronously and independently of the other agents. When the local functions are strongly convex with Lipschitz-continuous gradients, we show that the iterates at each agent converge to a neighborhood of the global minimum, where the neighborhood size depends on the degree of asynchrony in the multiagent network. When the agents work at the same rate, convergence to the global minimizer is achieved. Numerical experiments demonstrate that asynchronous gradient push can minimize the global objective faster than the state-of-the-art synchronous first-order methods, is more robust to failing or stalling agents, and scales better with the network size.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiyan Jiang
2. Wenpeng Zhang
3. Wenwu Zhu
4. Jinjie Gu

In Computer Networks and Communications

1. Jiyan Jiang
2. Wenpeng Zhang
3. Wenwu Zhu
4. Jinjie Gu

Citations in the corpus

• Asynchronous Decentralized Online Learning (2021, methodology): “Our mechanism allows the learner to perform model averaging instantaneously during the waiting period, which is nevertheless not supported if we simply apply previous stochastic algorithms [3, 23] to the online setting.”