Efficient Output Kernel Learning for Multiple Tasks

By Pratik Jawanpuria, Maksim Lapin, Matthias Hein, Bernt Schiele. 2015

37 citations recorded by the source corpus.

The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other. While previously the relationship between tasks had to be user-defined in the form of an output kernel, recent approaches jointly learn the tasks and the output kernel. As the output kernel is a positive semidefinite matrix, the resulting optimization problems are not scalable in the number of tasks as an eigendecomposition is required in each step. Using the theory of positive semidefinite kernels we show in this paper that for a certain class of regularizers on the output kernel, the constraint of being positive semidefinite can be dropped as it is automatically satisfied for the relaxed problem. This leads to an unconstrained dual problem which can be solved efficiently. Experiments on several multi-task and multi-class data sets illustrate the efficacy of our approach in terms of computational efficiency as well as generalization performance.

Subfields: Artificial Intelligence, Computational Mechanics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

Citations in the corpus

A Survey on Multi-Task Learning (2017, background): “074/-/Parkinson ( [86]) Explained Variance↑ 2.”
Forward and Reverse Gradient-Based Hyperparameter Optimization (2017, background): “In Table 4, we report comparative results obtained with two state-of-the-art multitask learning methods (Dinuzzo et al. (2011) and Jawanpuria et al. (2015)) on the CIFAR-10 dataset.”
Efficient Multitask Feature and Relationship Learning (2017, unknown): “…past decades (Adel et al., 2017, Argyriou et al., 2007, 2008, Caruana, 1997, Chen et al., 2011, 2012, Evgeniou and Pontil, 2004, Jacob et al., 2009, Jawanpuria et al., 2015, Kato et al., 2008, Li et al., 2015, Liu et al., 2009, Zhang and Schneider, 2010, Zhang and Yeung, 2010a, Zhao et al., 2017,…”
Deep Learning for Medical Anomaly Detection – A Survey (2020, methodology): “For instance, in [36] the authors purposed an efficient kernel learning structure for multiple tasks and applied this framework to regress Parkinson’s disease symptom scores.”