Choosing Transfer Languages for Cross-Lingual Learning

By Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma and 3 more. 2019

265 citations recorded by the source corpus.

Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages. However, given a particular task language, it is not clear which language to transfer from, and the standard strategy is to select languages based on ad hoc criteria, usually the intuition of the experimenter. Since a large number of features contribute to the success of cross-lingual transfer (including phylogenetic similarity, typological properties, lexical overlap, or size of available data), even the most enlightened experimenter rarely considers all these factors for the particular task at hand. In this paper, we consider this task of automatically selecting optimal transfer languages as a ranking problem, and build models that consider the aforementioned features to perform this prediction. In experiments on representative NLP tasks, we demonstrate that our model predicts good transfer languages much better than ad hoc baselines considering single features in isolation, and glean insights on what features are most informative for each different NLP tasks, which may inform future ad hoc selection even without use of our method.

Subfields: Artificial Intelligence, Natural Language Processing, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Graham Neubig
  2. Sebastian Ruder
  3. Mohit Bansal
  4. Ivan Vulic
  5. Luke S. Zettlemoyer
  6. Samuel R. Bowman
  7. Noah A. Smith
  8. Yulia Tsvetkov

In Natural Language Processing

  1. Graham Neubig
  2. Sebastian Ruder
  3. Orhan Firat
  4. Yulia Tsvetkov
  5. Samuel R. Bowman
  6. Ivan Vulic
  7. Melvin Johnson
  8. Mohit Bansal

In Machine Translation

  1. Graham Neubig
  2. Yulia Tsvetkov
  3. Anoop Kunchukuttan
  4. Heike Adel
  5. Sebastian Ruder
  6. Melvin Johnson
  7. Anders Søgaard
  8. Ivan Vulic

Citations in the corpus