Inferring latent task structure for Multitask Learning by Multiple Kernel Learning

By Christian Widmer, Nora C. Toussaint, Yasemin Altun, Gunnar Rätsch. 2010

81 citations recorded by the source corpus.

BackgroundThe lack of sufficient training data is the limiting factor for many Machine Learning applications in Computational Biology. If data is available for several different but related problem domains, Multitask Learning algorithms can be used to learn a model based on all available information. In Bioinformatics, many problems can be cast into the Multitask Learning scenario by incorporating data from several organisms. However, combining information from several tasks requires careful consideration of the degree of similarity between tasks. Our proposed method simultaneously learns or refines the similarity between tasks along with the Multitask Learning classifier. This is done by formulating the Multitask Learning problem as Multiple Kernel Learning, using the recently published q-Norm MKL algorithm.ResultsWe demonstrate the performance of our method on two problems from Computational Biology. First, we show that our method is able to improve performance on a splice site dataset with given hierarchical task structure by refining the task relationships. Second, we consider an MHC-I dataset, for which we assume no knowledge about the degree of task relatedness. Here, we are able to learn the task similarities ab initio along with the Multitask classifiers. In both cases, we outperform baseline methods that we compare against.ConclusionsWe present a novel approach to Multitask Learning that is capable of learning task similarity along with the classifiers. The framework is very general as it allows to incorporate prior knowledge about tasks relationships if available, but is also able to identify task similarities in absence of such prior information. Both variants show promising results in applications from Computational Biology.

Subfields: Artificial Intelligence, Molecular Biology, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Optimization & Training

Citations in the corpus

An Overview of Multi-task Learning (2018, background): “…and Yang 39 populations [104], construction of personalized brain–computer interfaces [105], MHC-I binding prediction [106], splice-site prediction [106], protein subcellular location prediction [107], Alzheimer’s disease assessment scale cognitive subscale [108], prediction of cognitive outcomes…”
A Survey on Multi-Task Learning (2017, methodology): “In [186], two multi-task multi-kernel methods are proposed for MHC-I binding prediction and splice-site prediction.”
A Convex Feature Learning Formulation for Latent Task Structure Discovery (2012, methodology): “Landmine A benchmark multi-task classification dataset used in Xue et al. (2007); Zhang & Schneider (2010). It contains examples collected from various landmine fields.”
Hierarchical Multitask Structured Output Learning for Large-scale Sequence Segmentation (2011, background): “, [1]) assuming a cluster relationship [11] or a hierarchy [6, 23, 13] between tasks.”
Contextual Gaussian Process Bandit Optimization (2011, methodology): “The context similarity was obtained using the hamming distance between amino acids in the binding pocket [11] (see Figure 2(c)), and we used the Gaussian kernel on the extracted features.”
A Survey of Transfer and Multitask Learning in Bioinformatics (2011, background): “org [35] investigated multi-task learning scenarios where there exists a latent structure shared across tasks.”