Distributed optimization for multi-task learning via nuclear-norm approximation
By David Mateos-Núñez, Jorge Cortés. 2015
16 citations recorded by the source corpus.
Abstract We exploit a variational characterization of the nuclear norm to extend the framework of distributed convex optimization to machine learning problems that focus on the sparsity of the aggregate solution. We propose two distributed dynamics that can be used for multi-task feature learning and recommender systems in scenarios with more tasks or users than features. Our first dynamics tackles a convex minimization on local decision variables subject to agreement on a set of local auxiliary matrices. Our second dynamics employs a saddle-point reformulation through Fenchel conjugation of quadratic forms, avoiding the computation of the inverse of the local matrices. We show the correctness of both coordination algorithms using a general analytical framework developed in our previous work that combines distributed optimization and subgradient methods for saddle-point problems.
Subfields: Artificial Intelligence, Computational Mechanics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Virginia Smith
- Ameet Talwalkar
- Qiang Yang
- Maziar Sanjabi
- Jiayu Zhou
- Mohammad Rostami
- Kaixiang Lin
- Inci M. Baytas
In Computational Mechanics
- Mohammad Rostami
- Ameet Talwalkar
- Kaixiang Lin
- Inci M. Baytas
- Virginia Smith
- Jiayu Zhou
- Chao-Kai Chiang
- Maziar Sanjabi
Citations in the corpus
- Federated Multi-Task Learning (2017, background): “While several recent works [1, 33, 51, 52] have considered the issue of distributed MTL training, the proposed methods do not allow for flexibility of communication versus computation.”
- Learning Transferable Knowledge Through Embedding Spaces (2019, methodology): “Only recently have MTL scenarios been investigated where the tasks are distributed [235; 249; 250; 251; 252; 253].”
- Privacy-Preserving Distributed Multi-Task Learning with Asynchronous Updates (2017, background): “Mateos-Núñez et al. [35] explored a separable convex optimization problem and a separable saddle-point reformulation which can be extended to distributed MTL framework.”