Gossip Algorithms for Distributed Signal Processing

By A. Dimakis, Soummya Kar, José M. F. Moura, Michael G. Rabbat, Anna Scaglione. 2010

896 citations recorded by the source corpus.

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This paper presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.

Subfields: Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

Decentralized Deep Learning with Arbitrary Communication Compression (2019, background): “It has been observed that gossip averaging can diverge (or not converge to the correct solution) in the presence of quantization noise (Xiao et al., 2005; Carli et al., 2007; Nedić et al., 2008; Dimakis et al., 2010; Carli et al., 2010b; Yuan et al., 2012).”
Decentralized Collaborative Learning of Personalized Models over Networks (2016, methodology): “On the other hand, models with low confidence are allowed large deviations: in the extreme case where agent i has very little or even no data (i.e., ci is negligible), its model is fully determined by the neighboring models.”
An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination (2012, background): “…generalized coordinates, Mi(qi) ∈ Rp×p is the symmetric positive-definite inertia matrix, Ci(qi, q̇i)q̇i ∈ Rp is the vector of Coriolis and centrifugal torques,gi(qi) is the vector of gravitational torques, andτi ∈ Rp is the vector of torques produced by the actuators associated with theith agent.”
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization (2014, background): “Examples and works include decentralized averaging [7,15,34], learning [9,22,26], estimation [1,2,16,18, 29], sparse optimization [19, 35], and low-rank matrix completion [20] problems.”
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization (2017, methodology): “een studied in [61]–[64], while a different consensus algorithm (so called push-sum method) have been considered in [44], [45]. A nonlinear gossip method is investigated in [65], while a survey paper [66] provides a detailed account of gossip algorithms for distributed averaging and their applicatio”
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization (2013, background): “The values of x̃ can stand for average temperature of a room [5], [6], frequen cy-domain occupancy of spectra [11], [12], states of a smart grid syste m [13], [14], and so on.”
Expander graph and communication-efficient decentralized optimization (2016, background): “uting are widely found in sensor network information processing, multiple-agent control, coordination, distributed machine learning, among others. Some important works include decentralized averaging [1], [2], [3], learning [4], [5], [6], estimation [7], [8], [9], [10], sparse optimization [11], [12”
A Method to Improve Consensus Averaging using Quantized ADMM (2019, methodology): “Gossip algorithms have been used widely to solve distributed averaging problems under bandwidth constraints and dynamic network topologies [12], [13], [15].”
Decentralized consensus optimization with asynchrony and delays (2016, methodology): “Related decentralized algorithms under different settings Our setting of asynchrony is different from randomized single activation, which is assumed for the randomized gossip algorithm [15], [16].”
Asynchronous Gradient Push (2018, neutral): “In general, many of the asynchronous multi-agent optimization algorithms in the literature make restrictive assumptions regarding the nature of the agent updates ( e.g. , sparse Poisson point process [28], randomized single activation [29], [30], randomized multi-activation [31]–[34]).”
Distributed Sparse Linear Regression (2010, methodology): “Tutorial treatments of related consensus-based distributed approaches can be found in [ 9 ] and [23].”
Adaptation, Learning, and Optimization over Networks (2014, neutral): “, [18, 26, 32, 46, 84, 87, 128, 138, 139, 175, 176, 186, 199, 205, 209, 215, 225, 242, 243, 266, 268]; (c) diffusion strategies — see, e.”
Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip (2019, methodology): “3 Faster method for average consensus Background Average consensus (AC) is a fundamental problem in distributed computing and multi-agent systems [42, 16].”
Push-Sum Distributed Dual Averaging for convex optimization (2012, background): “ca Consensus algorithms have been analyzed mostly in wireless network settings [5] and provide an elegant distributed solution for coordinating the nodes.”
Adaptive Networks (2014, methodology): “(9)], [90, eq.”
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization (2014, background): “Examples and works include decentralized averaging [7, 15, 34], learning”