Decentralized RLS With Data-Adaptive Censoring for Regressions Over Large-Scale Networks

By Zifeng Wang, Zheng Yu, Qing Ling, Dimitris Berberidis, Georgios B. Giannakis. 2016

21 citations recorded by the source corpus.

The deluge of networked data motivates the development of algorithms for computation- and communication-efficient information processing. In this context, three data-adaptive censoring strategies are introduced to considerably reduce the computation and communication overhead of decentralized recursive least-squares solvers. The first relies on alternating minimization and the stochastic Newton iteration to minimize a network-wide cost, which discards observations with small innovations. In the resultant algorithm, each node performs local data-adaptive censoring to reduce computations while exchanging its local estimate with neighbors so as to consent on a network-wide solution. The communication cost is further reduced by the second strategy, which prevents a node from transmitting its local estimate to neighbors when the innovation it induces to incoming data is minimal. In the third strategy, not only transmitting, but also receiving estimates from neighbors is prohibited when data-adaptive censoring is in effect. For all strategies, a simple criterion is provided for selecting the threshold of innovation to reach a prescribed average data reduction. The novel censoring-based (C)D-RLS algorithms are proved convergent to the optimal argument in the mean-root deviation sense. Numerical experiments validate the effectiveness of the proposed algorithms in reducing computation and communication overhead.

Subfields: Computational Mechanics, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Qing Ling
2. Gang Wu
3. Anit Kumar Sahu
4. Soummya Kar
5. Yi Yu
6. Haiquan Zhao
7. Dragana Bajovic
8. Wei Xu

In Computer Networks and Communications

1. Qing Ling
2. Gang Wu
3. Anit Kumar Sahu
4. Angelia Nedić
5. Ali H. Sayed
6. Soummya Kar
7. Zhi Tian
8. Wei Xu

Citations in the corpus

• Robust Distributed Diffusion Recursive Least Squares Algorithms With Side Information for Adaptive Networks (2018, methodology): “Following this algorithm, to reduce computation and communication costs, its variants were presented by censoring observations with small innovations [34].”
• Communication-Efficient Distributed Strongly Convex Stochastic Optimization: Non-Asymptotic Rates. (2018, methodology): “However, the communication savings in [26] are a constant proportion with respect to a method which utilizes all communications at all times, thereby not improving the order of the convergence rate.”
• Communication-Censored ADMM for Decentralized Consensus Optimization (2019, methodology): “Related to the communication-censoring strategy in COCA, data-adaptive computation-censoring is a powerful tool to reduce the computation cost of big data processing over networks [36].”