When does deep multi-task learning work for loosely related document classification tasks?

By Emma Kerinec, Chloé Braud, Anders Søgaard. 2018

7 citations recorded by the source corpus.

This work aims to contribute to our understanding of when multi-task learning through parameter sharing in deep neural networks leads to improvements over single-task learning. We focus on the setting of learning from loosely related tasks, for which no theoretical guarantees exist. We therefore approach the question empirically, studying which properties of datasets and single-task learning characteristics correlate with improvements from multi-task learning. We are the first to study this in a text classification setting and across more than 500 different task pairs.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Adam Trischler
2. Tu Vu
3. Mohit Iyyer
4. Jianfeng Gao
5. Mohit Bansal
6. Yoshua Bengio
7. C. Pal
8. Tong Wang

Citations in the corpus

• Exploring and Predicting Transferability across NLP Tasks (2020, background): “However, manually designing such features is time-consuming and may not generalize well across classes of problems (Kerinec et al., 2018).”
• Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data (2020, background): “…from one task to another, several works have proposed carefully choosing which tasks to train as an intermediate step in NLP before single task fine-tuning (Bingel & Søgaard, 2017; Kerinec et al., 2018; Wang et al., 2019a; Standley et al., 2019; Pruksachatkun et al., 2020; Phang et al., 2018).”