Should All Cross-Lingual Embeddings Speak English?

By Antonios Anastasopoulos, Graham Neubig. 2019

34 citations recorded by the source corpus.

Most of recent work in cross-lingual word embeddings is severely Anglocentric. The vast majority of lexicon induction evaluation dictionaries are between English and another language, and the English embedding space is selected by default as the hub when learning in a multilingual setting. With this work, however, we challenge these practices. First, we show that the choice of hub language can significantly impact downstream lexicon induction zero-shot POS tagging performance. Second, we both expand a standard English-centered evaluation dictionary collection to include all language pairs using triangulation, and create new dictionaries for under-represented languages. Evaluating established methods over all these language pairs sheds light into their suitability for aligning embeddings from distant languages and presents new challenges for the field. Finally, in our analysis we identify general guidelines for strong cross-lingual embedding baselines, that extend to language pairs that do not include English.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Percy Liang
3. Sebastian Gehrmann
4. Mohit Bansal
5. Luke S. Zettlemoyer
6. Jianfeng Gao
7. Yejin Choi
8. Sebastian Ruder

In Natural Language Processing

1. Graham Neubig
2. Sebastian Gehrmann
3. Thibault Sellam
4. Percy Liang
5. Mohit Bansal
6. Orhan Firat
7. Ankur P. Parikh
8. Pengfei Liu

In Representation & Self-Supervised Learning

1. Graham Neubig
2. Melvin Johnson
3. Aditya Siddhant
4. Orhan Firat
5. Junjie Hu
6. Noah Constant
7. Pengfei Liu
8. Jinlan Fu

Citations in the corpus

• The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “clude English at all, which we hope reduces the dependence of the modeling approaches on anglocentric pretraining (Anastasopoulos and Neubig, 2020).”
• Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages (2020, background): “Most cross-lingual representation learning techniques are Anglo-centric in their de-sign (Anastasopoulos and Neubig, 2019).”
• XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation (2021, background): “Zero-shot crosslingual transfer additionally introduces a bias towards the source language (Søgaard et al., 2018; Anastasopoulos and Neubig, 2020).”