Multi-task and Lifelong Learning of Kernels

By Anastasia Pentina, Shai Ben-David. 2015

64 citations recorded by the source corpus.

We consider a problem of learning kernels for use in SVM classification in the multi-task and lifelong scenarios and provide generalization bounds on the error of a large margin classifier. Our results show that, under mild conditions on the family of kernels used for learning, solving several related tasks simultaneously is beneficial over single task learning. In particular, as the number of observed tasks grows, assuming that in the considered family of kernels there exists one that yields low approximation error on all tasks, the overhead associated with learning such a kernel vanishes and the complexity converges to that of learning when this good kernel is given to the learner.

Subfields: Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

A Survey on Multi-Task Learning (2017, background): “In [220], Pentina and Ben-David present a generalization bound for learning”
Kernel Continual Learning (2021, background): “Kernels have been shown to be effective for incremental and multi-task learning with support vector machines (Diehl & Cauwen-berghs, 2003; Pentina & Ben-David, 2015).”
Regret Bounds for Lifelong Learning (2016, methodology): “set Gof representations. Further applications of this algorithm which may be studied within our framework are deep neural networks and kernel learning. In the latter case, which has been addressed by Pentina and Ben-David (2015) in the learning-to-learn setting, g: X!Zis a feature map to a reproduci”
Domain Generalization by Marginal Transfer Learning (2017, methodology): “In the context of (reproducing) kernel methods, this has been studied under the form of learning a linear transform or projection in kernel space (Maurer, 2009), and learning the kernel itself (Pentina and Ben-David, 2015).”
Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning (2016, background): “Generalization performance analysis in life-long learning has been investigated by [59, 13, 12, 55] and [54].”
The Extreme Value Machine (2015, background): “Pentina and Ben-David [20] lay out a cogent”
Lifelong Learning with Weighted Majority Votes (2016, background): “These settings were theoretically analyzed in a series of works [18, 19, 23, 21] that have demonstrated that information transfer can lead to provable sample complexity reductions compared to solving each task independently.”
On Localized Discrepancy for Domain Adaptation (2020, background): “Multitask learning problem concentrates on how to share knowledge between different tasks for better overall results Maurer (2006); Pentina and Ben-David (2015); Maurer et al. (2016); Pentina and Lampert (2017); Wu et al. (2020).”
Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint. (2020, methodology): “Kronecker factored Laplace approximation The basic idea of the Kronecker factored Laplace approximation algorithm by Ritter et al. (2018) is similar to that of EWC, with the key difference being that it uses Kronecker product approximation of the Fisher information matrix (Martens and Grosse, 2015; ”