Improved Neural Machine Translation with SMT Features

By Wei He, Zhongjun He, Hua Wu, Haifeng Wang. 2016

123 citations recorded by the source corpus.

Neural machine translation (NMT) conducts end-to-end translation with a source language encoder and a target language decoder, making promising translation performance. However, as a newly emerged approach, the method has some limitations. An NMT system usually has to apply a vocabulary of certain size to avoid the time-consuming training and decoding, thus it causes a serious out-of-vocabulary problem. Furthermore, the decoder lacks a mechanism to guarantee all the source words to be translated and usually favors short translations, resulting in fluent but inadequate translations. In order to solve the above problems, we incorporate statistical machine translation (SMT) features, such as a translation model and an n-gram language model, with the NMT model under the log-linear framework. Our experiments show that the proposed method significantly improves the translation quality of the state-ofthe-art NMT system on Chinese-to-English translation tasks. Our method produces a gain of up to 2.33 BLEU score on NIST open test sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tagged Back-Translation (2019, methodology): “Various meth-ods for using target-side LMs have proven effective for NMT (He et al., 2016; G¨ulc¸ehre et al., 2017), but have tended to be less successful than back-translation – for example, G¨ulc¸ehre et al. (2017) report under +0.5 B LEU over their base-line on EnDe newstest14, whereas Edunov et ”
Guiding Neural Machine Translation with Retrieved Translation Pieces (2018, methodology): “A number of methods have been proposed to ameliorate these problems, including methods that incorporate symbolic knowledge such as discrete translation lexicons (Arthur et al., 2016; He et al., 2016; Chatterjee et al., 2017) and phrase tables (Zhang et al.”
Correcting Length Bias in Neural Machine Translation (2018, methodology): “A variety of methods, like minimum error rate training (Och, 2003; He et al., 2016), are possible, but keeping with the globally-normalized negative log-likelihood, we obtain, for the constant word reward, the gradient: If we approximate the expectation using the mode of the distribution, we get…”
On NMT Search Errors and Model Errors: Cat Got Your Tongue? (2019, background): “, 2016) or word rewards (He et al., 2016): Length normalization gives an advantage to longer hypotheses by dividing the score by the sentence length, while a word reward directly violates monotonicity as it rewards each word with a positive value.”
STACL: Simultaneous Translation with Integrated Anticipation and Controllable Latency (2018, background): “Moreover, limited memory forces human interpreters to routinely omit source content (He et al., 2016), and the best of them can only retain about 60% of source material.”
The University of Cambridge’s Machine Translation Systems for WMT18 (2018, background): “…2018), MBR-based formalisms (Stahlberg et al., 2017a, 2018; Igle-sias et al., 2018), NMT assisting SMT (Junczys-Dowmunt et al., 2016b; Du and Way, 2017), and SMT assisting NMT (Niehues et al., 2016; He et al., 2016; Long et al., 2016; Wang et al., 2017; Dahlmann et al., 2017; Zhou et al., 2017).”
Neural Machine Translation: A Review (2019, methodology): “to combine NMT scores with other features normally used in SMT (like language models) in a log-linear model [271, 303].”
If Beam Search Is the Answer, What Was the Question? (2020, background): “In the context of the degenerate behavior of these models, the beam search curse—a specific phenomenon where using a larger beam size leads to worse performance—has been analyzed by a number of authors (Koehn and Knowles, 2017; Murray and Chiang, 2018; Yang et al., 2018; Stahlberg and Byrne, 2019; J”
slimIPL: Language-Model-Free Iterative Pseudo-Labeling (2020, methodology): “• computer vision: graph-based methods [25, 26], generative modelling [27, 28], consistency-based methods [29, 30, 31, 32], and contrastive methods [33, 34, 35, 36]; • machine translation (MT): integration of a language model trained on monolingual data [37, 38, 39], backtranslation [40, 41, 42], sy”