Regularized multi--task learning

By Theodoros Evgeniou, Massimiliano Pontil. 2004

1,664 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Representation & Self-Supervised Learning

Citations in the corpus

An Overview of Multi-Task Learning in Deep Neural Networks (2017, methodology): “A similar constraint for SVMs was also proposed by [Evgeniou and Pontil, 2004].”
An Overview of Multi-task Learning (2018, methodology): “Here s ( W t ) uses the regularizer proposed in [36] to enforce different columns in W t to be close to their average.”
Learning multiple visual domains with residual adapters (2017, methodology): “Earlier examples of this paradigm have focused on kernel methods [10, 1] and deep neural network (DNN) models [6].”
MetaICL: Learning to Learn In Context (2021, methodology): “…objective, MetaICL achieves substantial improvements even with smaller LMs. Meta-training via multi-task learning Our work is broadly inspired by a large body of work in meta-learning (Vilalta and Drissi, 2002; Finn et al., 2017) and multi-task learning (Evgeniou and Pontil, 2004; Ruder, 2017).”
Cross-Stitch Networks for Multi-task Learning (2016, background): “The term multi-task learning (MTL) itself has been broadly used [2, 14, 28, 42, 54, 55] as an umbrella term to include representation learning and selection [4, 13, 31, 37], transfer learning [39, 41, 56] etc.”
Semi-Cyclic Stochastic Gradient Descent (2019, background): “…structure (Ando & Zhang, 2005), shared deep features (e.g. Caruana, 1997), shared kernel or low nuclearnorm structure (e.g. Argyriou et al., 2008; Amit et al., 2007), low-norm additive structure (e.g. Evgeniou & Pontil, 2004), or graph-based relatedness (e.g. Evgeniou et al., 2005; Maurer, 2006).”
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs (2019, methodology): “This principle, known as graph regularization, is well-established in the multi-task learning literature (Evgeniou and Pontil, 2004; Maurer, 2006; Dhillon et al., 2011).”
A Survey on Multi-Task Learning (2017, methodology): “In [107], r(Wa) adopts the regularizer proposed in [73] which enforces the parameters of all the tasks to approach their average.”
Federated Multi-Task Learning (2017, background): “The ﬁrst (e.g., [13, 4, 10, 23]) assumes that a clustered, sparse, or low-rank structure between the tasks is known a priori .”
Personalized and Private Peer-to-Peer Machine Learning (2017, methodology): “Following previous work (see e.g., Evgeniou and Pontil, 2004; Vanhaesebrouck et al., 2017), we assume that the edge weights reflect a notion of “task relatedness”: the weight Wij between agents i and j tends to be large if the models minimizing their respective expected loss are similar.”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “There exists a large body of literature in machine learning that has modeled statistical heterogeneity via methods such as meta-learning [114] and multi-task learning [18, 37]; these ideas have been recently extended to the federated setting [24, 26, 35, 58, 106, 138].”
Taking Advantage of Multitask Learning for Fair Classification (2018, methodology): “A common approach to overcome such limitations is offered by Multitask Learning (MTL), see [5–7 13 18] and references therein.”
Towards a Neural Statistician (2016, methodology): “For examples see Lawrence and Platt [20] where share they priors between Gaussian processes, and Evgeniou and Pontil [5] where they take an SVM-like approach to share kernels.”
Overcoming Catastrophic Forgetting by Incremental Moment Matching (2017, methodology): “es weight-transfer. Explanation of Untuned Tuned Disjoint MNIST Experiment Hyperparam Hyperparam Accuracy Hyperparam Accuracy SGD [3] epoch per dataset 10 47.72 ( 0.11) 0.05 71.32 ( 1.54) L2-transfer [25] in (10) - - 0.05 85.81 ( 0.52) Drop-transfer pin (11) 0.5 51.72 ( 0.79) 0.5 51.72 ( 0.79) EWC [”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, methodology): “We formulate it in form of a L2 penalization over adaptive parameters of the representation [26, 22, 11]:”
Pre-Trained Models: Past, Present and Future (2021, methodology): “4, transfer learning can be categorized under four sub-settings, inductive transfer learning (Lawrence and Platt, 2004; Mihalkova et al., 2007; Evgeniou and Pontil, 2004), transductive transfer learning (Shimodaira, 2000; Zadrozny, 2004; Daume III and Marcu, 2006), self-taught learning (Raina et…”