Distributed Multi-Task Learning

By Jialei Wang, Mladen Kolar, Nati Srebro. 2015

79 citations recorded by the source corpus.

We consider the problem of distributed multi-task learning, where each machine learns a separate, but related, task. Specifically, each machine learns a linear predictor in high-dimensional space,where all tasks share the same small support. We present a communication-efficient estimator based on the debiased lasso and show that it is comparable with the optimal centralized method.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Networks and Communications, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Mohammad Rostami
3. Ameet Talwalkar
4. Yu Zhang
5. Virginia Smith
6. Eric Eaton
7. Sinno Jialin Pan
8. Aurélien Bellet

In Computational Mechanics

1. Qiang Yang
2. Yu Zhang
3. Mohammad Rostami
4. Ameet Talwalkar
5. Virginia Smith
6. Sinno Jialin Pan
7. Aurélien Bellet
8. Eric Eaton

In Computer Networks and Communications

1. Ameet Talwalkar
2. Virginia Smith
3. Qiang Yang
4. Yu Zhang
5. Maziar Sanjabi
6. Mohammad Rostami
7. Marc Tommasi
8. Paul Vanhaesebrouck

In Optimization & Training

1. Qiang Yang
2. Yu Zhang
3. Virginia Smith
4. Ameet Talwalkar
5. Maziar Sanjabi
6. Mohammad Rostami
7. Massimiliano Pontil
8. Peilin Zhao

Citations in the corpus

• An Overview of Multi-task Learning (2018, methodology): “In [83], a distributed algorithm is proposed based on a debiased lasso model and by learning one task in a machine, this al-gorithm achieves efficient communications.”
• A Survey on Multi-Task Learning (2017, neutral): “In [158], a communication-efficient distributed MTL”
• Federated Multi-Task Learning (2017, background): “While several recent works [1, 33, 51, 52] have considered the issue of distributed MTL training, the proposed methods do not allow for flexibility of communication versus computation.”
• Decentralized Collaborative Learning of Personalized Models over Networks (2016, methodology): “Lastly, the algorithms are centralized, aside from the distributed MTL of Wang et al. (2016) which is synchronous and relies on a central server.”
• Distributed variance regularized Multitask Learning (2016, background): “Very recently, in [Wang et al., 2015], the authors presented a distributed algorithm for group lasso multitask learning, addressing its statistical properties.”
• Distributed Multi-task Learning for Sensor Network (2017, methodology): “Our approach and [17] differ in the regularization schemes.”
• Multi-Agent Distributed Lifelong Learning for Collective Knowledge Acquisition (2017, methodology): “Similar to prior works [10, 22, 33], we use distributed optimization to tackle the collective lifelong learning problem.”
• Distributed multi-task classification: a decentralized online learning approach (2017, methodology): “Dinuzzo [16] first explored distributed MTL by performing information fusion with a server-client strategy, while later research [37,38] proposed distributed MTL algorithms by further restricting task relatedness through shared sparsity.”
• Distributed Multi-Task Relationship Learning (2017, background): “Different from prior solutions to distributed MTL, which are focused on the former category [5, 29, 30], our proposed DMTRL falls into the latter category.”
• Learning Transferable Knowledge Through Embedding Spaces (2019, methodology): “Only recently have MTL scenarios been investigated where the tasks are distributed [235; 249; 250; 251; 252; 253].”