Better Document-Level Machine Translation with Bayes’ Rule

By Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, Chris Dyer. 2019

48 citations recorded by the source corpus.

Abstract We show that Bayes’ rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents a compelling benefit because parallel documents are not always available. In our formulation, the posterior probability of a candidate translation is the product of the unconditional (prior) probability of the candidate output document and the “reverse translation probability” of translating the candidate output back into the source language. Our proposed model uses a powerful autoregressive language model as the prior on target language documents, but it assumes that each sentence is translated independently from the target to the source language. Crucially, at test time, when a source document is observed, the document language model prior induces dependencies between the translations of the source sentences in the posterior. The model’s independence assumption not only enables efficient use of available data, but it additionally admits a practical left-to-right beam-search algorithm for carrying out inference. Experiments show that our model benefits from using cross-sentence context in the language model, and it outperforms existing document translation approaches.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, background): “…Gu et al. (2016; 2017); Chen et al. (2018) explored the idea of using a small neural network to steer an LM. Yu et al. (2016), and more recently Yu et al. (2019); Yee et al. (2019); Ng et al. (2019), leveraged the Shannon Noisy Channel Theory (Shannon, 1948) for improving sequence-to-sequence…”
Data-to-text Generation with Macro Planning (2021, methodology): “(Yee et al., 2019; Yu et al., 2020), that is, by learning two different distributions: a conditional model that provides the probability of translating a paragraph plan to text and a language model that provides an unconditional estimate of the output (i.”
Encouraging Lexical Translation Consistency for Document-Level Neural Machine Translation (2021, background): “…outputs of a document more coherent, Voita et al. (2019a) pro-pose DocRepair trained on monolingual target language documents to correct the inconsistencies in sentence-level translation while Yu et al. (2020) train a context-aware language model to re-rank sentence-level translation candidates.”