A notion of task relatedness yielding provable multiple-task learning guarantees

By Shai Ben-David, Reba Schuller Borbely. 2008

131 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, background): “Multi-task learning requires some task relatedness [4] and it is common to pursue some automated identification of tasks which are related and should be reinforcing each other, e.”
A Survey on Multi-Task Learning (2017, methodology): “MTL Model Reference Analysis Tool Convergence rate Tasks from an environment [249], [250] VC dimension & Covering number O(1/ √ mn0) Task distributions can be transformed [254] VC dimension O(1/ √ mn0) Task clustering [49] VC dimension O(m ln(n0/m)/n0) Multi-task kernel classifier [257] Covering num”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “Ben-David & Borbely (2008) define two tasks as F -related if the data generation distribution of one task can be transformed into that of the other by some function in F .”
Model Zoo: A Growing Brain That Learns Continually (2021, background): “Larger the number of tasks n , smaller the d H ( n ) (Ben-David and Borbely, 2008).”
Taskonomy: Disentangling Task Transfer Learning (2018, methodology): “malized transfer afﬁnities using AHP (Analytic Hierarchy Process). IV. Find global transfer taxonomy using BIP (Binary Integer Program). to map a hypothesis for one task onto a hypothesis for another [7], through information-based approaches [60], or through modeling inductive bias [6]. For these gu”
Joint covariate selection and joint subspace selection for multiple classification problems (2010, background): “The general problem of jointly estimating models from multiple, related data sets is often referred to as “transfer learning” or “multi-task learning” in the machine learning literature (Maurer 2006; Ben-David and Schuller-Borbely 2008; Argyriou et al. 2008; Jebara 2004; Evgeniou and Pontil 2004; To”
Multi-Task Learning and Algorithmic Stability (2015, methodology): “Also based on the VC dimension, Ben-David and Borbely (2008) study the generalization bound for multi-task algorithms where the data distributions of any pair of tasks are assumed to be transformable.”
Cross-Domain Multitask Learning with Latent Probit Models (2012, background): “…the tasks have the same input and output domains and differ only in data distributions (Baxter, 2000; Bakker & Heskes, 2003; Argyriou et al., 2007; Ben-David & Borbely, 2008), a number of recent publications are beginning to break the limit of this assumption, in an attempt of extending MTL to a…”
Towards Knowledgeable Supervised Lifelong Learning Systems (2020, methodology): “Since: w = l∑ i=1 αiyiφ(xi) (28) and: w∗ = l∗∑ i=1 α∗i yiφ(xi) (29) and: l∗ ≤ l (30) For: αm = {α|0 α C},αb = {α|α = C}, α0 = {α|α = 0} (31) and: α∗m = {α ∗|0 α∗ C},α∗b = {α ∗|α∗ = C}, α∗0 = {α ∗|α∗ = 0} (32) With: |α∗m| ≥ |αm| (33) Then: ∆∗ ≥ ∆ (34) and, by Theorem 3 in Vapnik (1998) : V C-”
On the Theory of Transfer Learning: The Importance of Task Diversity (2020, background): “1 Related Work The utility of multitask learning methods was observed at least as far back as Caruana [1997]. In recent years, representation learning, transfer learning, and meta-learning have been the subject of extensive empirical investigation in the machine learning literature (see [Bengio et a”
Multi-Task Learning with Deep Neural Networks: A Survey (2020, background): “Still, many non-neural multi-task learning methods are motivated and justified by strong theory (Baxter, 2000; Ben-David and Borbely, 2008; Zhang, 2015; Lounici et al., 2009), but aside from a small pool of recent work (Shui et al.”
Operator-valued Kernels for Learning from Functional Response Data (2015, background): “There is a large literature on this subject, in particular Evgeniou and Pontil (2004); Jebara (2004); Ando and Zhang (2005); Maurer (2006); Ben-david and Schuller-Borbely (2008); Argyriou et al. (2008) and references therein.”
Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning (2016, background): “Also, in the context of domain adaptation, similar considerations are examined by [41, 43, 42, 21, 66, 44] and [22].”