Learning Sparse Task Relations in Multi-Task Learning

By Yu Zhang, Qiang Yang. 2017

48 citations recorded by the source corpus.

In multi-task learning, when the number of tasks is large, pairwise task relations exhibit sparse patterns since usually a task cannot be helpful to all of the other tasks and moreover, sparse task relations can reduce the risk of overfitting compared with the dense ones. In this paper, we focus on learning sparse task relations. Based on a regularization framework which can learn task relations among multiple tasks, we propose a SParse covAriance based mulTi-taSk (SPATS) model to learn a sparse covariance by using the ℓl regularization. The resulting objective function of the SPATS method is convex, which allows us to devise an alternating method to solve it. Moreover, some theoretical properties of the proposed model are studied. Experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Subfields: Artificial Intelligence, Computational Mechanics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Yu Zhang
3. Ying Wei
4. Mingsheng Long
5. Jianmin Wang
6. Dongrui Wu
7. Sinno Jialin Pan
8. Sebastian Ruder

In Computational Mechanics

1. Qiang Yang
2. Yu Zhang
3. Ying Wei

In Representation & Self-Supervised Learning

1. Qiang Yang
2. Yu Zhang
3. Ying Wei
4. Mingsheng Long
5. Jianmin Wang
6. Dit-Yan Yeung
7. Yu Zhang
8. Dongrui Wu

Citations in the corpus

• An Overview of Multi-task Learning (2018, methodology): “The MTRL method is generalized to multi-task boosting [46] and multi-label learning [47], where each label is treated as a task, and extended to learn sparse task relations in [48].”
• A Survey on Multi-Task Learning (2017, methodology): “For example, the methods in [73], [74], [75], [76], [77], which utilize a priori information on task relations, directly learn W and b by defining g(Σ) = 0, some task clustering methods [65], [69] identify task clusters by assuming that Σ has a block structure, several task relation learning methods”
• Learning to Multitask (2018, methodology): “The survey [34] has shown that the models proposed in [14, 13, 16, 32, 27, 33] can be formulated as problem (1) with different g ( · ) ’s, where the detailed connections between these works and problem (1) are put in the supplementary material for completeness.”