Multitask Diffusion Adaptation Over Networks

By Jie Chen, Cédric Richard, Ali H. Sayed. 2013

264 citations recorded by the source corpus.

Adaptive networks are suitable for decentralized inference tasks. Recent works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with l2-regularization. The stability and performance of the algorithm in the mean and mean-square error sense are analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to target localization and hyperspectral data unmixing.

Subfields: Computational Mechanics, Computer Networks and Communications, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Networks and Communications

In Media Technology

Citations in the corpus

DJAM: Distributed Jacobi Asynchronous Method for Learning Personal Models (2018, background): “Problem (1) is also referred to as multitask problem; this problem is solved in [17] for a more restricted class of personal losses than ours.”
Asynchronous Saddle Point Algorithm for Stochastic Optimization in Heterogeneous Networks (2019, background): “Alternative attempts to extend multi-agent optimization techniques to heterogeneously correlated problems have been considered in [23], [24] for special loss functions and correlation models, but the generic problem was online recently solved in [19] with a stochastic variant of the saddle point met”
Distributed multi-task classification: a decentralized online learning approach (2017, background): “Decentralized Distributed Learning – NS Previous research (Chen et al. 2014) of assigning each node to tackle one specific task, also known as Node Specific methods.”
Proximity Without Consensus in Online Multiagent Optimization (2016, background): “However, we allow for constraints to be generically chosen convex inequalities, rather than a specific Euclidean penalty, as in [25].”
Multi-Agent Distributed Lifelong Learning for Collective Knowledge Acquisition (2017, background): “Other approaches assume that processing power is distributed among the agents, which exchange information with their neighbors during the learning process [7].”
Learning Transferable Knowledge Through Embedding Spaces (2019, methodology): “Existing methods in the literature have mostly investigated special cases of this setting for distributed multi-task learning (MTL) [237; 238; 235].”