Excess risk bounds for multitask learning with trace norm regularization

By Massimiliano Pontil, Andreas Maurer. 2012

122 citations recorded by the source corpus.

Trace norm regularization is a popular method of multitask learning. We give excess risk bounds with explicit dependence on the number of tasks, the number of examples per task and properties of the data distribution. The bounds are independent of the dimension of the input space, which may be infinite as in the case of reproducing kernel Hilbert spaces. A byproduct of the proof are bounds on the expected norm of sums of random positive semidefinite matrices with subexponential moments.

Subfields: Applied Mathematics, Computational Mechanics, Mathematical Physics, Optimization & Training, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computational Mechanics

In Mathematical Physics

In Optimization & Training

In Statistical Learning Theory

Citations in the corpus

Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “Some recent theoretical works [18, 19, 26, 36, 20, 35, 9] provide generalization and sample complexity guarantees for certain formalizations of multi-task learning based on the existence of a good representation.”
Efficient Representations for Lifelong Learning and Autoencoding (2014, background): “This case has been considered in the “batch” setting in which one has data available for all target functions at the same time and therefore can solve a joint optimization problem [1, 19].”
The Benefit of Multitask Representation Learning (2015, methodology): “Speciﬁcally, we introduce the classes The D ’s appearing in the deﬁnition of H are also called dictionaries and the individual d k are called atoms (see Maurer et al., 2013).”
Incremental Learning-to-Learn with Statistical Guarantees (2018, background): “LTL also bears strong similarity to multitask learning (MTL) [11] and much work has been done on the theoretical study of both batch [2, 21] and online [12] multitask learning algorithms.”
Multilinear Multitask Learning (2013, background): “Copyright 2013 by the author(s). learning process, which leads to better generalization if the tasks are related (Ando and Zhang, 2005; Baxter, 2000; Maurer, 2009; Maurer and Pontil, 2012; Maurer et. al, 2013).”
Generalize Across Tasks: Efficient Algorithms for Linear Representation Learning (2019, background): “In contrast, the known bounds for trace norm constraints are restricted to observed tasks (see (Maurer and Pontil, 2013)).”
What Makes Multimodal Learning Better than Single (Provably) (2021, background): “[40, 7, 32, 10] have provided the sample complexity bounds in the special case of the linear feature and linear task mapping.”
Provable and Efficient Continual Representation Learning (2022, background): “Some of the earliest works towards this goal include Baxter (2000) and linear settings of Lounici et al. (2011); Pontil and Maurer (2013); Wang et al. (2016); Cavallanti et al. (2010).”
Operator-valued Kernels for Learning from Functional Response Data (2015, result): “In the case of vector-valued functions, the effort in this area has already produced several successful results, including Baxter (2000), Ando and Zhang (2005), Maurer (2006), and Maurer and Pontil (2013).”