Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM
By Chaouki Ben Issaid, Anis Elgabli, Jihong Park, Medhi Bennis. 2020
13 citations recorded by the source corpus.
In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored-and-Quantized Generalized GADMM (CQ-GGADMM), leverages the novel worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the benchmark schemes based on decentralized ADMM without censoring, quantization, and/or the worker grouping method of GADMM.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Jihong Park
- Medhi Bennis
- Sumudu Samarakoon
- Anis Elgabli
- Merouane Debbah
- Seong-Lyun Kim
- Mingzhe Chen
- Joongheon Kim
Citations in the corpus
- Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications (2020, background): “For a sparse bipartite graph, including ring network topologies, GADMM and its variants have been proposed for achieving the linear convergence under quantization, link censoring, and time-varying topologies [64], [97], [170].”