Does Neural Machine Translation Benefit from Larger Context?

By Sébastien Jean, Stanislas Lauly, Orhan Firat, Kyunghyun Cho. 2017

153 citations recorded by the source corpus.

We propose a neural machine translation architecture that models the surrounding text in addition to the source sentence. These models lead to better performance, both in terms of general translation quality and pronoun prediction, when trained on small corpora, although this improvement largely disappears when trained with a larger corpus. We also discover that attention-based neural machine translation is well suited for pronoun prediction and compares favorably with other approaches that were specifically designed for this task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Machine Translation

Citations in the corpus

Context-Aware Neural Machine Translation Learns Anaphora Resolution (2018, background): “…models trained in an end-to-end fashion, an alternative is to directly provide additional context to an NMT system at training time and hope that it will succeed in inducing relevant predictive features (Jean et al., 2017; Wang et al., 2017; Tiedemann and Scherrer, 2017; Baw-den et al., 2018).”
Search Engine Guided Non-Parametric Neural Machine Translation (2017, background): “More recently, this kind of larger context translation has been applied to cross-sentential modeling, where the translation of a current sentence is done with respect to previous sentences (Jean et al., 2017; Wang et al., 2017).”
Exploiting Cross-Sentence Context for Neural Machine Translation (2017, background): “As additional attention leads to more computational cost, they can only incorporate limited information such as single preceding sentence in Jean et al. (2017).”
Pretrained Language Models for Document-Level Neural Machine Translation (2019, background): “…Machine Translation (NMT) is drawing more attention from researchers studying on incorporating contexts into translation models (Wang et al., 2017; Jean et al., 2017; Tiedemann and Scherrer, 2017; Voita et al., 2018; Zhang et al., 2018; Miculicich et al., 2018; Kuang et al., 2018; Tu et al.,…”
A Comprehensive Survey of Grammatical Error Correction (2021, background): “Integrating cross-sentence context is common in the researches about neural machine translation [62, 77, 133], where different sizes of contexts are considered.”
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (2023, background): “…related to work on contextual machine translation, where source/target context is used to improve sentence-level machine translation [House, 2006, Jean et al., 2017, Wang et al., 2017, Tiedemann and Scherrer, 2017, Kuang et al., 2018, Agrawal et al., 2018, Miculicich et al., 2018, Zhang et al.,…”
Search Engine Guided Neural Machine Translation (2018, background): “More recently, this kind of larger context translation has been applied to cross-sentential modeling, where the translation of a current sentence is done with respect to previous sentences(Jean et al. 2017; Wang et al. 2017).”
Encouraging Lexical Translation Consistency for Document-Level Neural Machine Translation (2021, background): “, previous sentences (Jean et al., 2017; Wang et al., 2017; Zhang et al., 2018; Bawden et al., 2018; Voita et al., 2018, 2019b; Yang et al., 2019), or entire document (Maruf and Haffari, 2018; Mace and Servan, 2019; Maruf et al.”
Better Document-Level Machine Translation with Bayes’ Rule (2019, background): “Existing work in the area of context-aware NMT typically adapts the MT sys-tem to take additional context as input, either a few previous sentences (Jean et al., 2017; Wang et al., 2017; Tu et al., 2018; Voita et al., 2018; Zhang et al., 2018; Werlen et al., 2018) or the full document (Haffari and…”
Coreference and Coherence in Neural Machine Translation: A Study Using Oracle Experiments (2018, methodology): “They either have an extra RNN-based encoder (Jean et al., 2017; Bawden et al., 2018) or work on very long sentences (Tiedemann and Scherrer, 2017).”
Data augmentation using back-translation for context-aware neural machine translation (2019, methodology): “…translation ( NMT ) models in sentence-level translation, context-aware NMT models have been studied to further boost the quality of translation (Jean et al., 2017; Tiedemann and Scherrer, 2017; Wang et al., 2017; Bawden et al., 2018; Voita et al., 2018; Maruf and Haffari, 2018; Miculicich et…”
Findings of the 2017 DiscoMT Shared Task on Cross-lingual Pronoun Prediction (2017, methodology): “Yet, it is clear that it has access to larger context compared to phrase-based SMT models, potentially spanning multiple sentences, which can improve pronoun translation (Jean et al., 2017a).”
Improving the Transformer Translation Model with Document-Level Context (2018, methodology): “While document-level NMT has attracted increasing attention from the community in the past two years (Jean et al., 2017; Kuang et al., 2017; Tiedemann and Scherrer, 2017; Wang et al., 2017; Maruf and Haffari, 2018; Bawden et al., 2018; Tu et al., 2018; Voita et al., 2018), to the best of our knowled”
Document-Level Neural Machine Translation with Hierarchical Attention Networks (2018, methodology): “Recent studies (Tiedemann and Scherrer, 2017; Jean et al., 2017; Wang et al., 2017; Tu et al., 2018) have demonstrated that adding contextual information to the NMT model improves the general translation performance, and more importantly, improves the coherence and cohesion of the translated text…”
Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches (2017, methodology): “Jean et al. (2017) propose a novel larger-context neural machine translation model based on the recent works on larger-context language modelling (Wang and Cho, 2016) and employ the method to model the surrounding text in addition to the source sentence.”