Cross-Lingual Transfer Learning for Question Answering

By Chia-Hsuan Lee, Hung-yi Lee. 2019

26 citations recorded by the source corpus.

Deep learning based question answering (QA) on English documents has achieved success because there is a large amount of English training examples. However, for most languages, training examples for high-quality QA models are not available. In this paper, we explore the problem of cross-lingual transfer learning for QA, where a source language task with plentiful annotations is utilized to improve the performance of a QA model on a target language task with limited available annotations. We examine two different approaches. A machine translation (MT) based approach translates the source language into the target language, or vice versa. Although the MT-based approach brings improvement, it assumes the availability of a sentence-level translation system. A GAN-based approach incorporates a language discriminator to learn language-universal feature representations, and consequentially transfer knowledge from the source language. The GAN-based approach rivals the performance of the MT-based approach with fewer linguistic resources. Applying both approaches simultaneously yield the best results. We use two English benchmark datasets, SQuAD and NewsQA, as source language data, and show significant improvements over a number of established baselines on a Chinese QA task. We achieve the new state-of-the-art on the Chinese QA dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Diyi Yang
3. Derek Tam
4. Colin Raffel
5. Jiaao Chen
6. Shayne Longpre
7. Hannaneh Hajishirzi
8. Luke S. Zettlemoyer

In Computer Vision and Pattern Recognition

1. Mohit Bansal
2. Derek Tam
3. Diyi Yang
4. Colin Raffel
5. Shayne Longpre
6. Jiaao Chen
7. Yi Lu
8. Joachim Daiber

In Natural Language Processing

1. Mohit Bansal
2. Diyi Yang
3. Colin Raffel
4. Jiaao Chen
5. Derek Tam
6. Shayne Longpre
7. Yi Lu
8. Joachim Daiber

In Question Answering & Reading Comprehension

1. Mohit Bansal

Citations in the corpus

• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, background): “…languages, including cross-lingual transfer learning which transfers models from resource-rich to resource-poor languages (Do and Gaspers, 2019; Lee and Lee, 2019; Schuster et al., 2019), few/zero-shot learning (Johnson et al., 2017; Blissett and Ji, 2019; Pham et al., 2019; Abad et al., 2020)…”
• MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2020, background): “Due to the lack of nonEnglish training data, most other approaches build on these, or leverage some other form of transfer learning (Cui et al., 2019a; Hsu et al., 2019; Lee and Lee, 2019; Kumar et al., 2019).”