Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

By Shyam Upadhyay, Kai-Wei Chang, Matt Taddy, Adam Tauman Kalai, James Y. Zou. 2017

27 citations recorded by the source corpus.

Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense wor d embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields comparable performance to a state of the art monolingual model trained on five times more training data.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “In a recent work, [27] provided an innovative way to address this deficit.”
Debiasing Gender biased Hindi Words with Word-embedding (2019, background): “Applications of pre-trained word-embeddings include various downstream natural language processing (NLP) tasks such as machine translation [1, 2, 3], sentiment analysis [4], dialogue generation [5], contextualized word representations [6, 7].”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, methodology): “This bilingual model was extended by Upadhyay, Chang, Zou, Taddy, and Kalai (2017) to a multilingual setting, in order to better beneﬁt from multilingual distributional information.”
Learning Gender-Neutral Word Embeddings (2018, background): “…have been made by leveraging semantic lexicons and morphology (Luong et al., 2013; Faruqui et al., 2014), disambiguating multiple senses (ˇSuster et al., 2016; Arora et al., 2018; Upadhyay et al., 2017), and modeling contextualized information by deep neural networks (Peters et al., 2018).”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “[27] provided an innovative way to address this deficit.”