On randomized distributed coordinate descent with quantized updates

By M. Gamal, Mostafa El Gamal, Lifeng Lai. 2016

5 citations recorded by the source corpus.

In this paper, we study the randomized distributed coordinate descent algorithm with quantized updates. In the literature, the iteration complexity of the randomized distributed coordinate descent algorithm has been characterized under the assumption that machines can exchange updates with an infinite precision. We consider a practical scenario in which the messages exchange occurs over channels with finite capacity, and hence the updates have to be quantized. We derive sufficient conditions on the quantization error such that the algorithm with quantized update still converge. We further verify our theoretical results by running an experiment, where we apply the algorithm with quantized updates to solve a linear regression problem.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Optimization & Training

Citations in the corpus

Federated Learning: Strategies for Improving Communication Efficiency (2016, background): “Incremental, randomized and distributed optimization algorithms can be similarly analyzed in a quantized updates setting [16, 8, 7].”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “Incremental, randomized and distributed optimization algorithms can be similarly analyzed in a quantized updates setting [144, 66, 64].”
Fault Tolerance in Iterative-Convergent Machine Learning (2018, background): “Mania et al. (2015) and El Gamal and Lai (2017) also consider a model similar to (2), however, perturbations are only added to the gradients.”