Unsupervised Multilingual Word Embeddings
By Xilun Chen, Claire Cardie. 2018
141 citations recorded by the source corpus.
Multilingual Word Embeddings (MWEs) represent words from multiple languages in a single distributional vector space. Unsupervised MWE (UMWE) methods acquire multilingual embeddings without cross-lingual supervision, which is a significant advantage over traditional supervised approaches and opens many new possibilities for low-resource languages. Prior art for learning UMWEs, however, merely relies on a number of independently trained Unsupervised Bilingual Word Embeddings (UBWEs) to obtain multilingual embeddings. These methods fail to leverage the interdependencies that exist among many languages. To address this shortcoming, we propose a fully unsupervised framework for learning MWEs that directly exploits the relations between all language pairs. Our model substantially outperforms previous approaches in the experiments on multilingual word translation and cross-lingual word similarity. In addition, our model even beats supervised approaches trained with cross-lingual resources.
Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Antonios Anastasopoulos
- Graham Neubig
- Minheng Ni
- Felix Stahlberg
- Hinrich Schütze
- Pengfei Liu
- Timo Schick
- Yulia Tsvetkov
In Natural Language Processing
- Antonios Anastasopoulos
- Minheng Ni
- Graham Neubig
- Felix Stahlberg
- Yulia Tsvetkov
- Orhan Firat
- Pengfei Liu
- Hinrich Schütze
In Representation & Self-Supervised Learning
Citations in the corpus
- Should All Cross-Lingual Embeddings Speak English? (2019, methodology): “The main approaches are: • Multilingual adversarial training with pseudo-randomized refinement (Chen and Cardie, 2018, MAT+MPSR ): a generalization of the adversarial approach of Zhang et al. (2017); Conneau et al. (2018) to multiple languages, also combined with an iterative refinement procedure.”
- CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP (2020, background): “It can be viewed as a collection of binary prediction problems by using a distinct estimator for each slot-value pair [Chen et al., 2018].”