Adversarial Multi-task Learning for Text Classification

By Pengfei Liu, Xipeng Qiu, Xuanjing Huang. 2017

662 citations recorded by the source corpus.

Neural network models have shown their promising opportunities for multi-task learning, which focus on learning the shared layers to extract the common and task-invariant features. However, in most existing approaches, the extracted shared features are prone to be contaminated by task-specific features or the noise brought by other tasks. In this paper, we propose an adversarial multi-task learning framework, alleviating the shared and private latent feature spaces from interfering with each other. We conduct extensive experiments on 16 different text classification tasks, which demonstrates the benefits of our approach. Besides, we show that the shared knowledge learned by our proposed model can be regarded as off-the-shelf knowledge and easily transferred to new tasks. The datasets of all 16 tasks are publicly available at http://nlp.fudan.edu.cn/data/.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

BAM! Born-Again Multi-Task Networks for Natural Language Understanding (2019, background): “Much of the recent work for NLP has centered on neural architecture design: e.g., ensuring only beneﬁcial information is shared across tasks (Liu et al., 2017; Ruder et al., 2019) or arranging tasks in linguistically-motivated hierarchies (Søgaard and Goldberg, 2016; Hashimoto et al., 2017; Sanh et…”
A Survey on Multi-Task Learning (2017, background): “Moreover, in [21], each task can learn its specific feature representation to increase the expressive power of the whole model.”
Latent Multi-Task Architecture Learning (2017, methodology): “Liu, Qiu, and Huang (2017) use a similar technique for sentiment analysis.”
Sluice networks: Learning what to share between loosely related tasks (2017, methodology): “We build on recent work trying to learn where to split merged networks [21], as well as work trying to learn how best to combine private and shared subspaces [5, 18].”
Generative Adversarial Network in Medical Imaging: A Review (2018, methodology): “Success of adversarial text classiﬁcation (Liu et al., 2017b) also prompts potential utility of GANs in improving performance of such systems for automatic MR protocol generation from free-text clinical indications (Sohn et al., 2017).”
Counter-Interference Adapter for Multilingual Machine Translation (2021, background): “The parameter-sharing among different languages may be a potential cause of negative interference (Liu et al., 2017; Wang et al., 2020).”
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (2019, background): “…address this issue of learning long-term dependencies, which has proven to be effective in a wide range of NLP tasks, such as text classiﬁcation (Liu et al., 2017, 2016b), semantic matching (Rockt¨aschel et al., 2015; Liu et al., 2016a), text summarization (Rush et al., 2015) and machine…”
Is GPT-3 a Good Data Annotator? (2022, neutral): “…(Devlin et al., 2019; Peters et al., 2018; Yang et al., 2019; Raffel et al., 2020; Liu et al., 2021b), multi-task learning (Glo-rot et al., 2011; Liu et al., 2017), semi-supervised learning (Miyato et al., 2016), and few-shot learning (Deng et al., 2019; He et al., 2021; Qin and Joty, 2022a).”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “specific features [25, 144, 194].”
UniT: Multimodal Multitask Learning with a Unified Transformer (2021, neutral): “[52, 17, 33, 50, 10]), or multimodal areas ( e.g .”
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, methodology): “Our MT-Tri model lends itself well to shared-private models such as those proposed recently (Liu et al., 2017; Kim et al., 2017), which extend upon (Ganin et al.”
Meta Multi-Task Learning for Sequence Modeling (2018, methodology): “• ASP-MTL: Proposed by (Liu, Qiu, and Huang 2017), us- ing adversarial training method on PSP-MTL.”
Deep Learning--based Text Classification (2020, background): “[175] extend adversarial training to the multi-task learning framework for TC [36], aiming to alleviate the taskindependent (shared) and task-dependent (private) latent feature spaces from interfering with each other.”
A Survey of Domain Adaptation for Neural Machine Translation (2018, methodology): “Recently, some of the adversarial methods began to be introduced into some NLP tasks (Liu et al., 2017; Chen et al., 2017b) and NMT (Britz et al.”
BERT for Joint Intent Classification and Slot Filling (2019, methodology): “, 2016), adversarial multi-task learning (Liu et al., 2017), and others.”
Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination (2018, background): “, 2017b), text classification (Liu et al., 2017a), and image classification (Liu et al.”
Cross-topic Argument Mining from Heterogeneous Sources (2018, methodology): “For mtl, we use a shared–private model (Liu et al., 2017), which showed promising results for text classification and word segmentation (Chen et al.”