Multi-Task Learning of Pairwise Sequence Classification Tasks over Disparate Label Spaces

By Isabelle Augenstein, Sebastian Ruder, Anders Søgaard. 2018

74 citations recorded by the source corpus.

We combine multi-task learning and semi-supervised learning by inducing a joint embedding space between disparate label spaces and learning transfer functions between label embeddings, enabling us to jointly leverage unlabelled data and auxiliary, annotated datasets. We evaluate our approach on a variety of tasks with disparate label spaces. We outperform strong single and multi-task baselines and achieve a new state of the art for aspect-based and topic-based sentiment analysis.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Zhiyuan Liu
3. Maosong Sun
4. Sebastian Ruder
5. Chaojun Xiao
6. Shuai Wang
7. Zixuan Ke
8. Mohit Bansal

Citations in the corpus

• Latent Multi-Task Architecture Learning (2017, neutral): “In contrast to previous studies on MTL (Martínez Alonso and Plank 2017; Bingel and Søgaard 2017; Augenstein, Ruder, and Søgaard 2018), our model also consistently outperforms single-task learning.”
• A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, neutral): “Unlike (Hashimoto et al. 2017) and other previous work (Katiyar and Cardie 2017; Bekoulis et al. 2018; Augenstein, Ruder, and Søgaard 2018), we do not learn label embed-dings, meaning that the (supervised) output/prediction of a layer is not directly fed to the following layer through an embedding…”
• Forward and Backward Knowledge Transfer for Sentiment Classification (2019, background): “Our work is also closely related to transfer learning (TL) (Pan and Yang, 2010; Blitzer et al., 2007; Andreevskaia and Bergler, 2008; Bollegala et al., 2011; He et al., 2011; Li et al., 2012, 2013; Xia and Zong, 2011; Pan et al., 2010; Wu et al., 2009, 2017; Augenstein et al., 2018).”
• Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, background): “Recent NLP conferences witnessed a “tsunami” of deep learning papers (Manning, 2015), followed by what we call a multi-task learning “wave”: MTL has been successfully applied to a wide range of NLP tasks (Cohn and Specia, 2013; Cheng et al., 2015; Luong et al., 2015; Plank et al., 2016; Fang and Coh”
• Multilingual and Multi-Aspect Hate Speech Analysis (2019, background): “, 2016), sentiment analysis (Augenstein et al., 2018), and other tasks.”
• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, unknown): “, 2020), multi-task learning (Glorot et al., 2011; Liu et al., 2017; Augenstein et al., 2018), semi-supervised learning (Zhu, 2005; Chapelle et al.”
• Legal Judgment Prediction via Topological Learning (2018, background): “, 2017) or handing unlabeled data (Augenstein et al., 2018).”