Learning what to share between loosely related tasks

By Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, Anders Søgaard. 2017

41 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, methodology): “While “learning to share” approaches (Kang et al., 2011; Ruder et al., 2017; Platanios et al., 2018) are promising alternatives, a more straightforward solution could be implicitly increasing the per-task capacity by increasing over-all model capacity.”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, methodology): “We selected the simple yet effective training method described in (Søgaard and Goldberg 2016; Ruder et al. 2017): after each parameter update, a task is randomly selected and a batch of the dataset attached to this task is also sampled at random to train the model.”
Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language (2018, background): “Some recent works have suggested speciﬁc approaches to deal with this complexity: Ruder et al. (2017) propose to learn from data which parameters to share in MTL with sluice networks and show some gains on NLP tasks.”
On the Role of Weight Sharing During Deep Option Learning (2019, background): “…et al., 2015; Riemer et al., 2015; Misra et al., 2016; Rusu et al., 2016; Shazeer et al., 2017; Fernando et al., 2017; Rosenbaum et al., 2018; Ruder et al., 2017; Ramachandran & Le, 2018; Cases et al., 2019) that directly addresses catastrophic forgetting, increasing representation stability…”
Legal Judgment Prediction via Topological Learning (2018, background): “Ruder et al. (2017) introduce a model which can decide the amount of sharing between different NLP tasks.”
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, background): “Recent NLP conferences witnessed a “tsunami” of deep learning papers (Manning, 2015), followed by what we call a multi-task learning “wave”: MTL has been successfully applied to a wide range of NLP tasks (Cohn and Specia, 2013; Cheng et al., 2015; Luong et al., 2015; Plank et al., 2016; Fang and Coh”
AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning (2019, methodology): “Multi-task learning has been applied to a wide range of natural language processing problems (Luong et al., 2015; Pasunuru and Bansal, 2017; Hashimoto et al., 2017; Ruder et al., 2017b; Kaiser et al., 2017; McCann et al., 2018).”