Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
By Zihan Liu, Yan Xu, Genta Indra Winata, Pascale Fung. 2019
21 citations recorded by the source corpus.
This paper describes CAiRE’s submission to the unsupervised machine translation track of the WMT’19 news shared task from German to Czech. We leverage a phrase-based statistical machine translation (PBSMT) model and a pre-trained language model to combine word-level neural machine translation (NMT) and subword-level NMT models without using any parallel data. We propose to solve the morphological richness problem of languages by training byte-pair encoding (BPE) embeddings for German and Czech separately, and they are aligned using MUSE (Conneau et al., 2018). To ensure the fluency and consistency of translations, a rescoring mechanism is proposed that reuses the pre-trained language model to select the translation candidates generated through beam search. Moreover, a series of pre-processing and post-processing approaches are applied to improve the quality of final translations.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Philipp Koehn
- Ondrej Bojar
- Graham Neubig
- Rico Sennrich
- Luke S. Zettlemoyer
- Yonatan Belinkov
- Sebastian Riedel
- Barry Haddow
Citations in the corpus
- How Decoding Strategies Affect the Verifiability of Generated Text (2019, methodology): “Findings of the Association for Computational Linguistics: EMNLP 2020, pages 223–235 November 16 - 20, 2020. c©2020 Association for Computational Linguistics 223”
- Findings of the 2019 Conference on Machine Translation (WMT19) (2019, background): “CAIRE (Liu et al., 2019)”
- ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic (2020, methodology): “English and Multilingual LMs. Pre-trained LMs exploiting a self-supervised objective with masking such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b) have revolutionized NLP. Multilingual versions of these models such as mBERT and XLM-RoBERTa (Conneau et al., 2020) were also…”