Unsupervised Cross-Domain Word Representation Learning

By Danushka Bollegala, Takanori Maehara, Ken-ichi Kawarabayashi. 2015

76 citations recorded by the source corpus.

Meaning of a word varies from one domain to another. Despite this important domain dependence in word semantics, existing word representation learning methods are bound to a single domain. Given a pair of \emph{source}-\emph{target} domains, we propose an unsupervised method for learning domain-specific word representations that accurately capture the domain-specific aspects of word semantics. First, we select a subset of frequent words that occur in both domains as \emph{pivots}. Next, we optimize an objective function that enforces two constraints: (a) for both source and target domain documents, pivots that appear in a document must accurately predict the co-occurring non-pivots, and (b) word representations learnt for pivots must be similar in the two domains. Moreover, we propose a method to perform domain adaptation using the learnt word representations. Our proposed method significantly outperforms competitive baselines including the state-of-the-art domain-insensitive word representations, and reports best sentiment classification accuracies for all domain-pairs in a benchmark dataset.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Pivot Based Language Modeling for Improved Neural Domain Adaptation (2018, background): “, 2016) which performed substantially worse than the DA baselines ((a)-(d)); (2) We tried to compare to (Bollegala et al., 2015) but, similarly to ZR17, failed to replicate their results; and (3) We replaced PBLM with an LSTM-LM, but the results substantially degraded.”
Multi-View Domain Adapted Sentence Embeddings for Low-Resource Unsupervised Duplicate Question Detection (2019, background): “Other meth-ods, which we will call in-training, encourage the alignment of embeddings during training (e.g., Bollegala et al. (2015); Yang et al. (2017)).”
Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains (2018, background): “Bollegala et al. (2015) create domain-speciﬁc embeddings for pivots and non-pivots with the constraint that the pivot representations are similar across domains.”
Lifelong Domain Word Embedding via Meta-Learning (2018, methodology): “To solve this problem and also the limited in-domain corpus size problem, cross-domain embeddings have been investigated [Bollegala et al. , 2015; Yang et al. , 2017; Bolle-gala et al. , 2017] via transfer learning [Pan and Yang, 2010].”
Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification (2016, background): “Another line of work aims to learn a low dimensional representation for each feature in both domains based on predicting its neighboring features (Yang and Eisenstein, 2015; Bollegala et al., 2015).”
Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction (2018, background): “The proposed embedding mechanism is related to cross domain embeddings (Bollegala et al., 2015, 2017) and domain-speciﬁc embeddings (Xu et al., 2018a,b).”
Memory, Show the Way: Memory Based Few Shot Word Representation Learning (2018, methodology): “This demonstrates the advantage of the memory as a transfer medium over the word-frequency based transfer of CRE and DAREP.”