Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation

By Marcin Junczys-Dowmunt. 2019

170 citations recorded by the source corpus.

This paper describes the Microsoft Translator submissions to the WMT19 news translation shared task for English-German. Our main focus is document-level neural machine translation with deep transformer models. We start with strong sentence-level baselines, trained on large-scale data created via data-filtering and noisy back-translation and find that back-translation seems to mainly help with translationese input. We explore fine-tuning techniques, deeper models and different ensembling strategies to counter these effects. Using document boundaries present in the authentic and synthetic parallel data, we create sequences of up to 1000 subword segments and train transformer translation models. We experiment with data augmentation techniques for the smaller authentic data with document-boundaries and for larger authentic data without boundaries. We further explore multi-task training for the incorporation of document-level source language monolingual data via the BERT-objective on the encoder and two-pass decoding for combinations of sentence-level and document-level systems. Based on preliminary human evaluation results, evaluators strongly prefer the document-level systems over our comparable sentence-level system. The document-level systems also seem to score higher than the human references in source-based direct assessment.

Subfields: Artificial Intelligence, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Graham Neubig
  2. Quoc V. Le
  3. Luke S. Zettlemoyer
  4. Mohit Bansal
  5. Kyunghyun Cho
  6. Yoshua Bengio
  7. Jianfeng Gao
  8. Percy Liang

In Machine Translation

  1. Ankur Bapna
  2. Orhan Firat
  3. Wang Ling
  4. Laurent Sartran
  5. Marta Ruiz Costa-jussà
  6. Angela Fan
  7. William Chan
  8. Phil Blunsom

Citations in the corpus