Lattice-based Minimum Error Rate Training for Statistical Machine Translation

By Wolfgang Macherey, Franz Joseph Och, Ignacio Thayer, Jakob Uszkoreit. Conference on Empirical Methods in Natural Language Processing, 2008

359 citations recorded by the source corpus.

Minimum Error Rate Training (MERT) is an effective means to estimate the feature function weights of a linear model such that an automated evaluation criterion for measuring system performance can directly be optimized in training. To accomplish this, the training procedure determines for each feature function its exact error surface on a given set of candidate translations. The feature function weights are then adjusted by traversing the error surface combined over all sentences and picking those values for which the resulting error count reaches a minimum. Typically, candidates in MERT are represented as N-best lists which contain the N most probable translation hypotheses produced by a decoder. In this paper, we present a novel algorithm that allows for efficiently constructing and representing the exact error surface of all translations that are encoded in a phrase lattice. Compared to N-best MERT, the number of candidate translations thus taken into account increases by several orders of magnitudes. The proposed method is used to train the feature function weights of a phrase-based statistical machine translation system. Experiments conducted on the NIST 2008 translation tasks show significant runtime improvements and moderate BLEU score gains over N-best MERT.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2e74e29298f0f71694ac21958996d147191fe4b0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Batch Tuning Strategies for Statistical Machine Translation (2012, methodology): “• lb-mert : Lattice MERT (Machery et al., 2008) using unpruned lattices and aggregating only those paths on the line search’s upper envelope.”
11,001 New Features for Statistical Machine Translation (2009, background): “…introduced alternative discriminative training meth-ods (Tillmann and Zhang, 2006; Liang et al., 2006; Turian et al., 2007; Blunsom et al., 2008; Macherey et al., 2008), in which a recurring challenge is scalability: to train many features, we need many train-218 ing examples, and to train…”
Syntactically Guided Neural Machine Translation (2016, methodology): “For the NMT-HIERO+TUNING systems, lattice MERT (Macherey et al., 2008) is used to optimise λHiero and λNMT on the tuning sets.”
Domain Adaptation for Statistical Machine Translation with Monolingual Resources (2009, background): “As already observed in previous literature (Macherey et al., 2008), ﬁrst iterations of the tuning process produces very bad weights (even close to 0); this exceptional performance drop is attributed to an over-ﬁtting on the candidate repository.”
Tuning as Ranking (2011, methodology): “Lattice- and hypergraphbased variants of MERT (Macherey et al., 2008; Kumar et al., 2009) are more stable than traditional MERT, but also require significant engineering efforts.”
Mutual Information and Diverse Decoding Improve Neural Machine Translation (2016, methodology): “We separately trained two SEQ2SEQ models, one for p(y|x) and one for p(x|y).”
Models and Inference for Prefix-Constrained Machine Translation (2016, background): “However, it is well-known that n -best lists are poor approximations of MT structured output spaces (Macherey et al., 2008; Gimpel et al., 2013).”
The University of Cambridge’s Machine Translation Systems for WMT18 (2018, methodology): “We ﬁrst tried to use n - best or lattice MERT (Och, 2003; Macherey et al., 2008) to ﬁnd interpolation weights, but these techniques were not effective in our setting, possibly due to the lack of diversity and depth in n - best lists from standard beam search.”
The Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16 (2016, methodology): “The sparse tuple vector semiring enables us to optimize the λparameters with LMERT (Macherey et al., 2008) on a development set.”
Large Scale Parallel Document Mining for Machine Translation (2010, methodology): “Minimum error rate training (Macherey et al., 2008) under the BLEU criterion is used to optimize the feature function weights on development data consisting of the nv-dev2007 and news-dev2009 data sets provided by the organizers of the 2007 and 2009 WMT shared translation tasks 1 .”