On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning

By Ya Li, Xinmei Tian, Tongliang Liu, Dacheng Tao. 2018

45 citations recorded by the source corpus.

Multitask learning (MTL) aims to learn multiple tasks simultaneously through the interdependence between different tasks. The way to measure the relatedness between tasks is always a popular issue. There are mainly two ways to measure relatedness between tasks: common parameters sharing and common features sharing across different tasks. However, these two types of relatedness are mainly learned independently, leading to a loss of information. In this paper, we propose a new strategy to measure the relatedness that jointly learns shared parameters and shared feature representations. The objective of our proposed method is to transform the features of different tasks into a common feature space in which the tasks are closely related and the shared parameters can be better optimized. We give a detailed introduction to our proposed MTL method. Additionally, an alternating algorithm is introduced to optimize the nonconvex objection. A theoretical bound is given to demonstrate that the relatedness between tasks can be better measured by our proposed MTL algorithm. We conduct various experiments to verify the superiority of the proposed joint model and feature MTL method.

Subfields: Artificial Intelligence, Computational Mechanics, Representation & Self-Supervised Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yi Tay
2. Donald Metzler
3. Dara Bahri
4. Vinh Q. Tran
5. Dacheng Tao
6. Jai Gupta
7. Tal Schuster
8. Sebastian Ruder

In Computational Mechanics

1. Tongliang Liu
2. Dacheng Tao
3. Yi Tay
4. Donald Metzler
5. Sebastian Ruder
6. Tal Schuster
7. Fengxiang He
8. Gaurav Menghani

In Representation & Self-Supervised Learning

1. Dacheng Tao
2. Tongliang Liu
3. Yi Tay
4. Donald Metzler
5. Dara Bahri
6. Jai Gupta
7. Vamsi Krishna Aribandi
8. Sebastian Ruder

In Optimization & Training

1. Tongliang Liu
2. Dara Bahri
3. Dacheng Tao
4. Yi Tay
5. Donald Metzler
6. Vinh Q. Tran
7. Huaixiu Steven Zheng
8. Jianmo Ni

Citations in the corpus

• ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, background): “Outside of NLP, Zhang & Yeung (2010) introduced a convex optimization objective for learning task relationships, and Li et al. (2018) explore and exploit task relationships on a variety of diverse datasets.”
• Why ResNet Works? Residuals Generalize (2019, methodology): “These theoretical tools have been widely applied to analyze many algorithms (see, e.g., [31, 17, 36, 48]).”