Polyglot Prompt: Multilingual Multitask PrompTraining

By Jinlan Fu, See-Kiong Ng, Pengfei Liu. 2022

16 citations recorded by the source corpus.

This paper aims for a potential architectural breakthrough for multilingual learning and asks: could different tasks from different languages be modeled in a monolithic framework (without any task/language-specific module) ? The benefit of achieving this is not only that systems trained on low resources scenario can be assisted by more other languages and tasks, but opening new doors for future multilingual research. We approach this goal by developing a learning framework Polyglot Prompt , where prompting methods are introduced to learn a unified semantic space for different languages and tasks after proper multilingual prompt engineering . Experimentally, we perform a comprehensive evaluation on 6 tasks (topic classi-fication, sentiment classification, named entity recognition, question answering, natural language inference, summarization), 24 datasets, and 49 languages, which shows the efficacy of multilingual multitask prompting training and suggests several interesting observations. e.g., English prompts are polyglots since directly applying them to task samples in other languages could result in a better improvement. We also present an interpretable multi-lingual evaluation methodology and show how the proposed framework, multilingual multi-task prompt training, works. We release all datasets prompted in the best setting 1 and will release our code soon. 2

Subfields: Artificial Intelligence, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pengfei Liu
2. Jinlan Fu
3. See-Kiong Ng
4. Zhengbao Jiang
5. Graham Neubig
6. Yixin Liu
7. Weizhe Yuan
8. Xipeng Qiu

In Prompting & Reasoning

1. Pengfei Liu
2. Jinlan Fu
3. Zhengbao Jiang
4. Graham Neubig

Citations in the corpus

• GPTScore: Evaluate as You Desire (2023, background): “Choice of Prompt Template Prompt (Liu et al., 2021; Fu et al., 2022) templates define how task description, aspect definition, and context are organized.”
• BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting (2022, unknown): “Some work shows that multitask prompt-based training on a variety of tasks and English or translated prompts improves zero-shot cross-lingual and cross-task performance (Muennighoff et al., 2022; Fu et al., 2022).”