Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation

By Biao Zhang, Ankur Bapna, Rico Sennrich, Orhan Firat. 2021

97 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Luke S. Zettlemoyer
2. Michael Lewis
3. Quoc V. Le
4. Orhan Firat
5. Mohit Bansal
6. Kyunghyun Cho
7. Graham Neubig
8. Jianfeng Gao

In Computer Vision and Pattern Recognition

1. Orhan Firat
2. Quoc V. Le
3. Michael Lewis
4. Luc Van Gool
5. Ankur Bapna
6. Yang Feng
7. Dacheng Tao
8. Jianfeng Gao

Citations in the corpus

• Counter-Interference Adapter for Multilingual Machine Translation (2021, background): “Based on their design, Philip et al. (2020) proposed the monolingual adapter for easy extension to new pairs, and Zhang et al. (2021) introduced conditional language-specific routing strategy(CLSR) to enhance model capacity in language-specific representation.”
• Importance-based Neuron Allocation for Multilingual Neural Machine Translation (2021, background): “Such approaches involve inserting conditional languagespecific routing layer (Zhang et al., 2021), specific attention networks (Blackwood et al., 2018; Sachan and Neubig, 2018), adding task adapters (Bapna and Firat, 2019), and training model with different language clusters (Tan et al., 2019), and…”
• Tricks for Training Sparse Translation Models (2021, background): “However, in practice, this positive transfer is often mitigated by interference between languages (Arivazhagan et al., 2019; Tan et al., 2019; Zhang et al., 2020).”
• Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference (2021, background): “Allowing dedicated (unshared) parameters are known to be effective at mitigating interference (Zhang et al., 2021; Kong et al., 2021) and MoE variants are inherently learn such partitioning across languages/tasks.”
• Mixture of Languages: Improved Multilingual Encoders Through Language Grouping (2025, unknown): “…the well-established Mixture-of-Experts (MoE) approach (Shazeer et al., 2017; Lepikhin et al., 2020) to train MT systems (Elbayad et al., 2023; NLLB Team, 2024) or decoder-only LLMs (Du et al., 2022; Jiang et al., 2024), and a hybrid of the two with language-aware routing (Zhang et al., 2021).”