Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

By Xilun Chen, Ben Athiwaratkun, Claire Cardie, Kilian Q. Weinberger. Transactions of the Association for Computational Linguistics, 2016

330 citations recorded by the source corpus.

In recent years great success has been achieved in sentiment classification for English, thanks in part to the availability of copious annotated resources. Unfortunately, most languages do not enjoy such an abundance of labeled data. To tackle the sentiment classification problem in low-resource languages without adequate annotated data, we propose an Adversarial Deep Averaging Network (ADAN1) to transfer the knowledge learned from labeled data on a resource-rich source language to low-resource languages where only unlabeled data exist. ADAN has two discriminative branches: a sentiment classifier and an adversarial language discriminator. Both branches take input from a shared feature extractor to learn hidden representations that are simultaneously indicative for the classification task and invariant across languages. Experiments on Chinese and Arabic sentiment classification demonstrate that ADAN significantly outperforms state-of-the-art systems.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/aee4010d58fd1501f04ac556252ddb7bf4acd143

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, background): “[159] 2018 Aspect-level ADAN Softmax TARGET 5 classes: 42.”
Unsupervised Multilingual Word Embeddings (2018, methodology): “…have been successfully applied to various cross-lingual NLP tasks where annotated data is not available, such as cross-lingual text classiﬁcation (Chen et al., 2016), unsupervised BWE induction (Zhang et al., 2017; Lample et al., 2018b) and unsupervised machine translation (Lample et al., 2018a;…”
Adversarial Cross-Modal Retrieval (2017, methodology): “…our approach was inspired by the effectiveness of adversarial learning for various applications, like learning discriminative image features [17], or (un)supervised domain adaptation to enforce domain-invariant features [2, 6, 41], and regularizing correlation loss between cross-modal items [10].”
Balancing Training for Multilingual Neural Machine Translation (2020, background): “Multilingual models are trained to process different languages in a single model, and have been applied to a wide variety of NLP tasks such as text classification (Klementiev et al., 2012; Chen et al., 2018a), syntactic analysis (Plank et al.”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “Kim et al. (2017), Chen et al. (2018) and Lange et al. (2020c) worked with language-independent representations for cross-lingual transfer.”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, methodology): “Adversarial feature alignment has been utilized for UDA object detection in Chen et al. (2018c); Saito et al. (2019b); Sindagi et al. (2020); Hsu et al. (2020a); VS et al. (2021).”
Adversarial Removal of Demographic Attributes from Text Data (2018, background): “Several works apply adversarial training to textual data, in order to learn encoders that are invariant to some properties of the text (Chen et al., 2016; Conneau et al., 2017; Zhang et al., 2017; Xie et al., 2017).”
Adversarial Learning for Neural Dialogue Generation (2017, methodology): “Adversarial networks The idea of generative adversarial networks has enjoyed great success in computer vision (Radford et al., 2015; Chen et al., 2016a; Salimans et al., 2016).”
Domain-agnostic Question-Answering with Adversarial Training (2019, background): “classification (Chen et al., 2016; Liu et al., 2017; Chen and Cardie, 2018), and relation extraction (Wu et al.”
Multimodal Language Analysis with Recurrent Multistage Fusion (2018, methodology): “Each modality is represented by averaging temporal information through time, as shown for language-based sentiment analysis (Iyyer et al., 2015; Chen et al., 2016) and multimodal sentiment analysis (Abburi et al., 2016; Nojavanasghari et al., 2016; Zadeh et al., 2016; Morency et al., 2011).”
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond (2021, neutral): “…, UMSAB henceforth), as this task is by far the most studied problem in NLP in Twitter (cf., e.g., (Salameh et al., 2015; Zhou et al., 2016; Meng et al., 2012; Chen et al., 2018; Rasooli et al., 2018; Vilares et al., 2017; Barnes et al., 2019; Patwa et al., 2020; Barriere and Bal-ahur, 2020)).”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, neutral): “, 2016b) and adversarial learning (Karimi et al., 2020; Chen et al., 2018).”
Stacked Generative Adversarial Networks (2016, background): “The idea of training a discriminator in some representation space has been explored in the context of domain adaptation [12, 2, 24], and regularization of auto-encoders [38].”
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts (2019, background): “…(target domain) is available at training time, in addition to a labeled dataset from the training distribution (source domain) (Chen et al., 2011; Gong et al., 2012; Long et al., 2015; Ganin et al., 2016; Long et al., 2016; Tzeng et al., 2017; Hoffman et al., 2017; Csurka, 2017; Chen et al., 2018).”
Multinomial Adversarial Networks for Multi-Domain Text Classification (2018, methodology): “…to numerous tasks (e.g. image generation (Goodfellow et al., 2014), domain adaptation (Ganin et al., 2016), cross-lingual sentiment analysis (Chen et al., 2016)), we anticipate that MAN s will make a versatile machine learning framework with applications beyond the MDTC task studied in this…”
Cross-Lingual Transfer Learning for Question Answering (2019, methodology): “By considering the source and target languages to be two different domains, we propose using domain adversarial training (Chen et al., 2016; Kim et al., 2017) to cause the QA model to learn language-invariant latent representations, which encourages knowledge transfer be-tween languages.”
Calibration of Pre-trained Transformers (2020, neutral): “In contrast, BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) are large-scale, pre-trained language models with stacked, general purpose Transformer (Vaswani et al., 2017) layers. ness using sentiment analysis (Chen et al., 2018; Peng et al., 2018; Miller, 2019; Desai et al., 2019).”