Counter-Interference Adapter for Multilingual Machine Translation

By Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, Lei Li. 2021

78 citations recorded by the source corpus.

Developing a unified multilingual model has long been a pursuit for machine translation. However, existing approaches suffer from performance degradation -- a single multilingual model is inferior to separately trained bilingual ones on rich-resource languages. We conjecture that such a phenomenon is due to interference caused by joint training with multiple languages. To accommodate the issue, we propose CIAT, an adapted Transformer model with a small parameter overhead for multilingual machine translation. We evaluate CIAT on multiple benchmark datasets, including IWSLT, OPUS-100, and WMT. Experiments show that CIAT consistently outperforms strong multilingual baselines on 64 of total 66 language directions, 42 of which see above 0.5 BLEU improvement. Our code is available at \url{https://github.com/Yaoming95/CIAT}~.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Continual Prompt Tuning for Dialog State Tracking (2022, background): “Prompt tuning differs from embedding adapter (Zhu et al., 2021) that aims to address the multilingual embedding deficiency.”
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment (2023, methodology): “We classify additive fine-tuning into three groups: Adapter-based Fine-tuning [9], [14], [15], [16], [17], [18], [19], [20], [21], [22], [76], in which the adapter module is incorporated into the transformer, allowing for fine-tuning without modifying the pretrained parameters, Soft Prompt-based…”
Structured Pruning Adapters (2022, methodology): “…prior work has largely ignored the computational efﬁciency aspects of Adapters, which either increase the complexity of the network (He et al., 2022; Zhu et al., 2021; Li & Liang, 2021; Houlsby et al., 2019; Pfeiffer et al., 2021; Mahabadi et al., 2021) or leave it unaltered, at best, by utilizing…”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, methodology): “…P E F T M e t hod s f o r P L M s AdditiveFine-tuning Adapter-basedFine-tuning Adapter Design Serial Adapter [75], Parallel Adapter [67], CIAT [242], CoDA [101] Multi-task Adaptation AdapterFusion [145], AdaMix [180], PHA [226], AdapterSoup [29], MerA [70], Hyperformer [129] Soft…”