Multi-Task Learning and Algorithmic Stability
By Yu Zhang. 2015
41 citations recorded by the source corpus.
In this paper, we study multi-task algorithms from the perspective of the algorithmic stability. We give a definition of the multi-task uniform stability, a generalization of the conventional uniform stability, which measures the maximum difference between the loss of a multi-task algorithm trained on a data set and that of the multi-task algorithm trained on the same data set but with a data point removed in each task. In order to analyze multi-task algorithms based on multi-task uniform stability, we prove a generalized McDiarmid's inequality which assumes the difference bound condition holds by changing multiple input arguments instead of only one in the conventional McDiarmid's inequality. By using the generalized McDiarmid's inequality as a tool, we can analyze the generalization performance of general multi-task algorithms in terms of the multi-task uniform stability. Moreover, as applications, we prove generalization bounds of several representative regularized multi-task algorithms.
Subfields: Artificial Intelligence, Computational Mechanics, Computer Networks and Communications, Optimization & Training, Statistical Learning Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Qiang Yang
- Yu Zhang
- Mingsheng Long
- Jianmin Wang
- Dongrui Wu
- Sinno Jialin Pan
- Sebastian Ruder
- Philip S. Yu
In Computational Mechanics
In Computer Networks and Communications
In Optimization & Training
In Statistical Learning Theory
Citations in the corpus
- An Overview of Multi-task Learning (2018, neutral): “Thenthereare manystudiestoanalyzegeneralizationboundsofdif-ferentMTLapproaches,includinge.g. [7,134]for thefeaturetransformapproach,[135]forthefea-tureselectionapproach,[24,135–138]forthelow-rankapproach,[136]forthetask-relationlearning approach,and[138]forthedirtyapproach.”
- A Survey on Multi-Task Learning (2017, methodology): “Zhang [221] extends the conventional stability to the MTL setting by proving a generalized McDiarmids inequality and uses the proposed multi-task stability to analyze the trace norm regularization [49] in the low-rank approach and a decomposition model in [101] (i.”