Discriminative Training and Maximum Entropy Models for Statistical Machine Translation

By Franz Joseph Och, Hermann Ney. Annual Meeting of the Association for Computational Linguistics, 2002

1,234 citations recorded by the source corpus.

We present a framework for statistical machine translation of natural languages based on direct maximum entropy models, which contains the widely used source-channel approach as a special case. All knowledge sources are treated as feature functions, which depend on the source language sentence, the target language sentence and possible hidden variables. This approach allows a baseline machine translation system to be extended easily by adding new feature functions. We show that a baseline statistical machine translation system is significantly improved using this approach.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/37fadfb6d60e83e24c72d8a90da5644b39d6e8f0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Improved Neural Machine Translation with SMT Features (2016, neutral): “The widely used log-linear framework in SMT was introduced by Och and Ney (2002). p(ẽ|f̃) = exp( ∑m i=1 λiHi(f̃ , ẽ))∑ ẽ′ exp( ∑m i=1 λiHi(f̃ , ẽ ′)) (6) where, Hi(f̃ , ẽ) is a feature function and λi is the weight.”
Word Sense Disambiguation Improves Statistical Machine Translation (2007, background): “Hiero uses a general log-linear model (Och and Ney, 2002) where the weight of a derivation D for a particular source sentence and its translation is w(D) = ∏”
Hierarchical Phrase-Based Translation (2007, positive): “…(cid:1) P ( γ | α ) and P ( α | γ ), the latter of which is not found in the noisy-channel model, but has been previously found to be a helpful feature (Och and Ney 2002); (cid:1) the lexical weights P w ( γ | α ) and P w ( α | γ ), which estimate how well the words in α translate the words in…”
Improving Machine Translation Performance by Exploiting Non-Parallel Corpora (2005, methodology): “They provide indispensable training data for statistical machine translation (Brown et al. 1990; Och and Ney 2002) and have been found useful in research on automatic lexical acquisition (Gale and Church 1991; Melamed 1997), cross-language information retrieval (Davis and Dunning 1995; Oard 1997),…”
A Smorgasbord of Features for Statistical Machine Translation (2004, methodology): “…target sentences, we will choose the sentence with the highest probability: As an alternative to the often used source-channel approach (Brown et al., 1993), we directly model the posterior probability Pr ( e I 1 | f J 1 ) (Och and Ney, 2002) us-ing a log-linear combination of feature functions.”
Discriminative Reranking for Machine Translation (2004, methodology): “This approach used the same set of features as the alignment template approach in (Och and Ney, 2002).”
A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation (2007, methodology): “We use a phrase-based SMT system, Pharaoh, (Koehn et al., 2003; Koehn, 2004), which is based on a log-linear formulation (Och and Ney, 2002).”
Correcting Length Bias in Neural Machine Translation (2018, methodology): “Alternatively, we can adopt a two-tiered model, familiar from phrase-based translation (Och and Ney, 2002), ﬁrst training s and then training s while keeping the parameters of s ﬁxed, possibly on a smaller dataset.”
Encoding Source Language with Convolutional Neural Network for Machine Translation (2015, methodology): “Following Och and Ney (2002), we use a general loglinear framework.”
Towards Decoding as Continuous Optimisation in Neural Machine Translation (2017, methodology): “This is inspired by the direct and reverse feature functions commonly used in classical discriminative SMT (Och and Ney, 2002) which have been shown to offer some complementary benefits (although see Lopez and Resnik (2006)).”
A Deep Learning Approach to Machine Transliteration (2009, methodology): “Then, these models are fused in a log-linear model (Och and Ney, 2002), and we tune the model scaling factors discriminatively on the development n - best list using the downhill simplex algorithm.”
The Alignment Template Approach to Statistical Machine Translation (2004, background): “5.1 General Concept”
genCNN: A Convolutional Architecture for Word Sequence Prediction (2015, methodology): “The weights of all the features are tuned via MERT (Och and Ney, 2002) on NIST MT05, and tested on NIST MT06 and MT08.”
Mutual Information and Diverse Decoding Improve Neural Machine Translation (2016, methodology): “In that sense, our work is designed to incorporate into S EQ 2S EQ models features that have proved useful in phrase-based MT, like the reverse translation probability or sentence length (Och and Ney, 2002; Shen et al., 2010; Devlin et al., 2014).”
Hope and Fear for Discriminative Training of Statistical Translation Models (2012, background): “The most common way to do this has been to generate then-best derivations according to the model and to choose the one with the lowest loss (Och and Ney, 2002).”