Expander graph and communication-efficient decentralized optimization

By Yat Tin Chow, Wei Shi, Tianyu Wu, Wotao Yin. 2016

33 citations recorded by the source corpus.

In this paper, we discuss how to design the graph topology to reduce the communication complexity of certain algorithms for decentralized optimization. Our goal is to minimize the total communication needed to achieve a prescribed accuracy. We discover that the so-called expander graphs are near-optimal choices. We propose three approaches to construct expander graphs for different numbers of nodes and node degrees. Our numerical results show that the performance of decentralized optimization is significantly better on expander graphs than other regular graphs.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling (2019, neutral): “, t(∆) = ∆) in this paper, as considered in previous works (Tsianos et al., 2012; Chow et al., 2016; Lian et al., 2017b) and also observed in experimental result Figure 1(b).”
Compressed Distributed Gradient Descent: Communication-Efficient Consensus over Networks (2018, background): “[20] studied the tradeoff between communication requirements and prescribed accuracy.”
Nested Distributed Gradient Methods with Adaptive Quantized Communication (2019, methodology): “Our work is closely related to a few lines of research that we delineate below: 1) distributed first-order primal algorithms [9]–[11], [29]–[32]: methods that use only gradient information, 2) nested [11], [33]–[35]: methods that decompose the communication and computation steps, 3) communication ef”
Balancing Communication and Computation in Distributed Optimization (2017, background): “Closely related to our work are a few very recent contributions that incorporate communication considerations in the design of distributed algorithms [51], [52], [53], [54], [55], [56].”
Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges (2022, background): “• Measure of robustness, flexibility, fault tolerance, cost, or security [11] Partially Connected Networks • Define the classification in star, ring or random structured networks [52], [53]”