Quantized consensus ADMM for multi-agent distributed optimization

By Shengyu Zhu, Mingyi Hong, Biao Chen. 2015

60 citations recorded by the source corpus.

This paper considers multi-agent distributed optimization with quantized communication which is needed when inter-agent communications are subject to finite capacity and other practical constraints. To minimize the global objective formed by a sum of local convex functions, we develop a quantized distributed algorithm based on the alternating direction method of multipliers (ADMM). Under certain convexity assumptions, it is shown that the proposed algorithm converges to a consensus within log1+η Ω iterations, where η > 0 depends on the network topology and the local objectives, and O is a polynomial fraction depending on the quantization resolution, the distance between initial and optimal variable values, the local objectives, and the network topology. We also obtain a tight upper bound on the consensus error which does not depend on the size of the network.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Medhi Bennis
2. Anis Elgabli
3. Jihong Park
4. Chaouki Ben Issaid
5. Qing Ling
6. Lifeng Lai
7. M. Gamal
8. Gang Wu

In Computer Networks and Communications

1. Jihong Park
2. Medhi Bennis
3. Anis Elgabli
4. Chaouki Ben Issaid
5. Qing Ling
6. Gang Wu
7. Vaneet Aggarwal
8. Zhi Tian

Citations in the corpus

• GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning (2019, background): “…decreasing communication payload sizes is one popular solution, which is enabled by gradient quantization (Suresh et al., 2017), model parameter quantization (Zhu et al., 2016; Sriranga et al., 2019), and model output exchange for large-sized models via knowledge distillation (Jeong et al., 2018).”
• On randomized distributed coordinate descent with quantized updates (2016, background): “ADMM was considered in [3], [4], [5] to handle the communication bottleneck.”
• Communication-Censored ADMM for Decentralized Consensus Optimization (2019, background): “However, these algorithms cannot guarantee exact convergence to the consensual optimal solution, and the consensus error is caused by the quantization error [31], [32].”
• Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM (2020, background): “ralized (Bernstein et al., 2018; Suresh et al., 2017; Sun et al., 2019; Vogels et al., 2019; Alistarh et al., 2017; Horvath et al., 2019) and decentralized network topologies (Sriranga et al., 2019; Zhu et al., 2016; Koloskova et al., 2019; Gao and Huang, 2020; Elgabli et al., 2020b). Alternatively,”
• Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning (2019, unknown): “To reduce the bandwidth/power usage per channel use, decreasing communication payload sizes is one popular solution, which is enabled by gradient quantization [42], model parameter quantization [41], [43], and model output exchange for large-sized models via knowledge distillation [14].”
• ADMM Based Privacy-Preserving Decentralized Optimization (2017, background): “A quantized analysis of the ε-ball is usually notoriously involved and hence we refer interested readers to [59] which is dedicated to this problem.”