No Language Left Behind: Scaling Human-Centered Machine Translation

By Nllb team, Marta Ruiz Costa-jussà, James Cross, Onur cCelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht and 29 more. 2022

1,884 citations recorded by the source corpus.

Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today. However, such efforts have coalesced around a small subset of languages, leaving behind the vast majority of mostly low-resource languages. What does it take to break the 200 language barrier while ensuring safe, high quality results, all while keeping ethical considerations in mind? In No Language Left Behind, we took on this challenge by first contextualizing the need for low-resource language translation support through exploratory interviews with native speakers. Then, we created datasets and models aimed at narrowing the performance gap between low and high-resource languages. More specifically, we developed a conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Critically, we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200, and combined human evaluation with a novel toxicity benchmark covering all languages in Flores-200 to assess translation safety. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art, laying important groundwork towards realizing a universal translation system. Finally, we open source all contributions described in this work, accessible at https://github.com/facebookresearch/fairseq/tree/nllb.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model (2024, methodology): “To compare the models we have used 240 sentences for each supported language from FLORES+ [31].”
A Survey of Large Language Models (2023, neutral): “6TB - - T0 [28] Oct-2021 11 T5 X 512 TPU v3 27 h X CodeGen [76] Mar-2022 16 - - - 577B tokens - X GPT-NeoX-20B [77] Apr-2022 20 - - - 825GB - 96 40G A100 - X Tk-Instruct [78] Apr-2022 11 T5 X 256 TPU v3 4 h X UL2 [79] May-2022 20 - - - 1T tokens Apr-2019 512 TPU v4 - X X OPT [80] May-2022 175 - - - ”
Demystifying Prompts in Language Models via Perplexity Estimation (2022, methodology): “We use a set of 8 languages available in the NLLB translation model (Costa-jussà et al., 2022) that are relatively high resource and close to English, 2 to reduce the risk of noise.”
Towards Making the Most of ChatGPT for Machine Translation (2023, methodology): “We also evaluate the performance of ChatGPT in non-English-centric language pairs (since the pre-training process was dominated by the English tokens and the multilingual MT community argues it may harm the non-English-centric performance (Costa-jussà et al., 2022; Zan et al., 2022a, 2023).).”
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models (2023, background): “In the domain of translation, by harnessing large language models and their capabilities, it is possible to achieve more accurate and ﬂuent translations across various languages [17], including low-resource ones [18, 19, 20, 21].”
A Survey on Mixture of Experts in Large Language Models (2024, positive): “…unlocked extraordinary capabilities in a range of natural language understanding (NLU) and generation (NLG) tasks, including machine translation [24], [76], open-domain question answering [33], [90], code generation [26], [65], [67], [176], and mathematical problem-solving [26], [30], [67], [176].”
Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust (2022, methodology): “B LEURT (Sellam et al., 2020) is a learned metric that is fine-tuned to produce a DA for a given translation by encoding it jointly with its reference.”
Tool Learning with Foundation Models (2023, positive): “Following Toolformer, we use NLLB (Costa-jussà et al., 2022) as our translator and choose MLQA (Lewis et al., 2020a), a multilingual question answering benchmark, as the testbed.”
Toxicity in Multilingual Machine Translation at Scale (2022, methodology): “Toxicity detection methodology NLLB Team et al. (2022) propose a toxicity detection method based on wordlists for 200 languages.”
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation (2023, background): “Recent advancements in natural language processing (NLP), particularly the development of largescale language modeling techniques, have brought remarkable improvements in machine translation as well as in other NLP tasks (Fan et al., 2021; Kim et al., 2021; Costa-jussà et al., 2022).”
Exploring AI Ethics of ChatGPT: A Diagnostic Analysis (2023, methodology): “As machine translation is one of the most traditional tasks for multilingual understanding, we let Chat-GPT perform zero-shot machine translation using FLORES-200 [3] devtest set.”