A Convex Feature Learning Formulation for Latent Task Structure Discovery

By Pratik Jawanpuria, SakethaNath Jagarlapudi. 2012

29 citations recorded by the source corpus.

This paper considers the multi-task learning problem and in the setting where some relevant features could be shared across few related tasks. Most of the existing methods assume the extent to which the given tasks are related or share a common feature space to be known apriori. In real-world applications however, it is desirable to automatically discover the groups of related tasks that share a feature space. In this paper we aim at searching the exponentially large space of all possible groups of tasks that may share a feature space. The main contribution is a convex formulation that employs a graph-based regularizer and simultaneously discovers few groups of related tasks, having close-by task parameters, as well as the feature space shared within each group. The regularizer encodes an important structure among the groups of tasks leading to an efficient algorithm for solving it: if there is no feature space under which a group of tasks has close-by task parameters, then there does not exist such a feature space for any of its supersets. An efficient active set algorithm that exploits this simplification and performs a clever search in the exponentially large space is presented. The algorithm is guaranteed to solve the proposed formulation (within some precision) in a time polynomial in the number of groups of related tasks discovered. Empirical results on benchmark datasets show that the proposed formulation achieves good generalization and outperforms state-of-the-art multi-task learning algorithms in some cases.

Subfields: Artificial Intelligence, Computational Mechanics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Representation & Self-Supervised Learning

Citations in the corpus

An Overview of Multi-task Learning (2018, background): “Unlike [58] where each level involves a subset of tasks, a multi-level task-clustering method is proposed in [34] to cluster all the tasks at each level based on a structurally sparse regularizer (cid”
A Survey on Multi-Task Learning (2017, methodology): “Another way to relate different component matrices is to use a non-decomposable regularizer as [107] did, which is slightly different from problem (26) in terms of the regularizer.”
Learning Sparse Task Relations in Multi-Task Learning (2017, methodology): “2007), devising some regularizer inspired by the k-means clustering in (Jacob, Bach, and Vert 2008), integer programming in (Kang, Grauman, and Sha 2011), group-sparsity regularizers in (Jawanpuria and Nath 2012; Han and Zhang 2015a) and so on.”
Hierarchical Regularization Cascade for Joint Learning (2013, background): “…work investigating the sharing of information at different levels either assume that the structure of sharing is known, or solve the hard problem of clustering the tasks into sharing groups (Torralba et al., 2007; Kang et al., 2011; Jawanpuria & Nath, 2012; Kim & Xing, 2010; Zhao et al., 2011).”
Efficient Output Kernel Learning for Multiple Tasks (2015, methodology): “We compare the following algorithms: Single task learning (STL), multi-task methods learning the output kernel matrix (MTL [16], CMTL [12], MTRL [9]) and approaches that learn both input and output kernel matrices (MTFL [11], GMTL [10]).”