Convex Learning of Multiple Tasks and their Structure

By Carlo Ciliberto, Youssef Mroueh, Tomaso A. Poggio, Lorenzo Rosasco. 2015

70 citations recorded by the source corpus.

Reducing the amount of human supervision is a key problem in machine learning and a natural approach is that of exploiting the relations (structure) among different tasks. This is the idea at the core of multi-task learning. In this context a fundamental question is how to incorporate the tasks structure in the learning problem. We tackle this question by studying a general computational framework that allows to encode apriori knowledge of the tasks structure in the form of a convex penalty; in this setting a variety of previously proposed methods can be recovered as special cases, including linear and non-linear approaches. Within this framework, we show that tasks and their structure can be efficiently learned considering a convex optimization problem that can be approached by means of block coordinate methods such as alternating minimization and for which we prove convergence to the global minimum.

Subfields: Artificial Intelligence, Computational Mechanics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Mingsheng Long
3. Jianmin Wang
4. Zhangjie Cao
5. Yu Zhang
6. Bernt Schiele
7. Philip S. Yu
8. Kate Saenko

In Computational Mechanics

1. Qiang Yang
2. Yu Zhang
3. Bernt Schiele
4. Matthias Hein
5. Maksim Lapin
6. Philip S. Yu
7. Pratik Jawanpuria
8. Jianmin Wang

Citations in the corpus

• A Survey on Multi-Task Learning (2017, background): “Problem (21) can also been interpreted from the perspective of reproducing kernel Hilbert spaces for vector-valued functions [85], [86], [87], [88].”
• Learning Multiple Tasks with Deep Relationship Networks (2015, methodology): “There are generally two categories of approaches: (1) multi-task feature learning, which learns a shared feature representation such that the distribution shift across different tasks can be reduced [1, 2, 6, 5, 16]; and (2) multi-task relationship learning, which explicitly models the task relation”
• Efficient Output Kernel Learning for Multiple Tasks (2015, methodology): “(b) Plot showing the factor by which FMTL2S outperforms OKL and ConvexOKL over the hyper-parameter range on various data sets.”
• Learning Multiple Tasks with Multilinear Relationship Networks (2015, methodology): “A common strategy used by previous methods is to use identity covariance for feature covariance [31, 8] and class covariance [2], which implicitly assumes independent features and classes and cannot capture the dependencies between them.”