Incremental Adaptive Strategies Over Distributed Networks

By Cássio Guimarães Lopes, Ali H. Sayed. 2007

753 citations recorded by the source corpus.

An adaptive distributed strategy is developed based on incremental techniques. The proposed scheme addresses the problem of linear estimation in a cooperative fashion, in which nodes equipped with local computing abilities derive local estimates and share them with their predefined neighbors. The resulting algorithm is distributed, cooperative, and able to respond in real time to changes in the environment. Each node is allowed to communicate with its immediate neighbor in order to exploit the spatial dimension while limiting the communications burden at the same time. A spatial-temporal energy conservation argument is used to evaluate the steady-state performance of the individual nodes across the entire network. Computer simulations illustrate the results.

Subfields: Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning (2019, methodology): “On the one hand, we broadly categorize distributed algorithms proposed in the literature into three types: consensus, diﬀusion, and incremental (Lopes and Sayed, 2007).”
Distributed optimization over time-varying directed graphs (2013, background): “Distributed optimization of a sum of convex functions has received a surge of interest in recent years [13], [15], [6], [11], [9], [10], [19], [12], [2], [16], [4].”
Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs (2014, methodology): “Our work here contributes to the growing literature on distributed methods for optimization over networks [27], [15], [31], [14], [20], [42], [18], [19], [33], [22], [5], [9], [21], [12], [39], [13].”
Diffusion LMS Strategies for Distributed Estimation (2010, background): “These include incremental LMS [3], [5], [6], incremental RLS [3], diffusion LMS [3], [7], and diffusion RLS [8], [9].”
Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis (2008, methodology): “It is important to remark though, that these assumptions do not compromise the spatial–temporal nature of the problem, neither its distributiveness [4].”
Asynchronous Incremental Stochastic Dual Descent Algorithm for Network Resource Allocation (2017, background): “Depending on the mode of communication among the nodes, distributed algorithms can be broadly classified into three categories, namely, diffusion, consensus, and incremental [27].”
Adaptive Networks (2014, background): “At this point, we can resort to a useful incremental substitution, which has been widely studied in the literature (see, e.g., [68]–[74] and [76]–[80]).”
Distributed Optimization Over Time-Varying Directed Graphs (2015, neutral): “2364096 Distributed optimization of a sum of convex functions has received a surge of interest in recent years [3], [5], [8], [11]– [13], [15], [23], [27].”
Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis (2010, methodology): “In this section, we analyze the mean-square performance of the CTA diffusion LMS algorithm with adaptive combiners using the energy conservation approach [9], [10], [23], [28], [31].”
Diffusion Strategies for Distributed Kalman Filtering and Smoothing (2010, methodology): “We refer to the measurement update of (12) as the incremental step [29] since the optimal local estimate is generated by incrementally incorporating data sequentially from the neighborhood.”
Multitask Diffusion Adaptation Over Networks (2013, background): “Although centralized strategies with a fusion center can beneﬁt more fully from information collected throughout the network but stored at a single point, in most cases, distributed strategies are more attractive to solve inference problems in a collaborative and autonomous manner.”
Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks (2011, methodology): “Therefore, in steady-state, we have lim i→∞ Ew̃i = 0 (94) Next, we determine an expression (rather than a bound) for the MSE.”
Distributed LMS for Consensus-Based In-Network Adaptive Processing (2009, methodology): “In this section, we introduce the D-LMS algorithm, first going through the algorithm construction and salient features of its operation.”
Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization (2018, background): “The works [28], [29] apply random walks in the context of adaptive networks and relies on stochastic gradients.”
Adaptation, Learning, and Optimization over Networks (2014, background): “Readers can refer to more detailed studies that appear, for example, in [30, 31, 38, 55, 109, 129, 157, 162, 173, 194, 195, 210, 211].”