11,001 New Features for Statistical Machine Translation

By David Chiang, Kevin Knight, Wei Wang. 2009

270 citations recorded by the source corpus.

We use the Margin Infused Relaxed Algorithm of Crammer et al. to add a large number of new features to two machine translation systems: the Hiero hierarchical phrase-based translation system and our syntax-based translation system. On a large-scale Chinese-English translation task, we obtain statistically significant improvements of +1.5 Bleu and + 1.1 Bleu, respectively. We analyze the impact of the new features and the performance of the learning algorithm.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Edinburgh’s Phrase-based Machine Translation Systems for WMT-14 (2014, background): “…et al., 2013a) with 4 count-based supportive features, sparse domain indicator, phrase length, and count bin features (Blunsom and Os-borne, 2008; Chiang et al., 2009), a distortion limit of 6, maximum phrase-length of 5, 100-best translation options, Minimum Bayes Risk decoding (Kumar and…”
Learning to Write with Cooperative Discriminators (2018, background): “statistical machine translation (Koehn et al., 2003; Och, 2003; Watanabe et al., 2007; Chiang et al., 2009) and remain common with neural machine translation, even when an extremely large amount of data is available (Wu et al.”
Fast and Robust Neural Network Joint Models for Statistical Machine Translation (2014, methodology): “…(Shen et al., 2010) • Trait features (Devlin and Matsoukas, 2012) • Factored source syntax (Huang et al., 2013) • 7 sparse feature types, totaling 50k features (Chiang et al., 2009) • LM adaptation (Snover et al., 2008) We also perform 1000-best rescoring with the following features: • 5-gram…”
Transfer Learning for Low-Resource Neural Machine Translation (2016, methodology): “Additionally, the SBMT models use thousands of sparsely-occurring, lexicalized syntactic features (Chiang et al., 2009).”
Deep RNNs Encode Soft Hierarchical Syntax (2018, background): “However, previous work has shown that syntactic information (in the form of either input features or supervision) is useful for a wide variety of NLP tasks (Punyakanok et al., 2005; Chiang et al., 2009), even in the neural setting (Aharoni and Goldberg, 2017; Chen et al., 2017).”
On the Impact of Various Types of Noise on Neural Machine Translation (2018, methodology): “…operation sequence model (OSM) (Durrani et al., 2013), sparse domain indicator, phrase length, and count bin features (Blunsom and Osborne, 2008; Chiang et al., 2009), a maximum phrase-length of 5, compact phrase table (Junczys-Dowmunt, 2012) minimum Bayes risk decoding (Kumar and Byrne,…”
English SLT and MT system description for the IWSLT 2013 evaluation (2013, background): “• limitation of phrase length to 5 • sparse domain indicator, lexical, phrase length, and count bin features [22]”
The NIST 2008 Metrics for machine translation challenge—overview, methodology, metrics, and results (2009, background): “The time to run a metric affects its usefulness for evaluation and also system development; in particular, training approaches that involve tuning to metrics, such as Och’s (2003) Minimum Error Rate Training (MERT) as well as Chiang’s approach ( Chiang et al. (2009) ), benefit from fast-running metr”
Edinburgh’s Machine Translation Systems for European Language Pairs (2013, methodology): “There was only a minimal change in the length ratio (”
Tuning as Ranking (2011, methodology): “…(cf. Chiang et al. (2009), Section 4.1) • Target word insertion features9 We used the following feature classes in SBMT extended scenarios only (cf. Chiang et al. (2009), Section 4.1):10 • Rule overlap features • Node count features 9For Chinese-English and Urdu-English SBMT these features…”
cdec: A Decoder, Alignment, and Learning Framework for Finite- State and Context-Free Translation Models (2010, methodology): “Further training pipelines are under development, including minimum risk training using a linearly decomposable approximation of BLEU (Li and Eisner, 2009), and MIRA training (Chiang et al., 2009).”
The JHU Machine Translation Systems for WMT 2016 (2016, background): “, 2013) with 4 count-based supportive features, sparse domain indicator, phrase length, and count bin features (Blunsom and Osborne, 2008; Chiang et al., 2009), a distortion limit of 6, maximum phrase-length of 5, 100-best translation options, compact phrase table (JunczysDowmunt, 2012) minimum Baye”
Syntactically Guided Neural Machine Translation (2016, background): “By contrast, Hiero source language parses completely cover the source sentence (Chiang et al., 2009).”
First- and Second-Order Expectation Semirings with Applications to Minimum-Risk Training on Translation Forests (2009, methodology): “We plan to incorporate more informative features described by Chiang et al. (2009).”
Direct Loss Minimization for Structured Prediction (2010, methodology): “After formulating our method we discovered that closely related methods have recently become popular for training machine translation systems (Liang et al., 2006; Chiang et al., 2009).”
Batch Tuning Strategies for Statistical Machine Translation (2012, methodology): “First employed in SMT by Watanabe et al. (2007), and reﬁned by Chiang et al. (2008; 2009), the Margin Infused Relaxed Algorithm (MIRA) employs a structured hinge loss: (2) where e ∗ i is an oracle derivation, and cost is de-ﬁned as i by a margin proportional to their BLEU differentials.”
Edinburgh SLT and MT system description for the IWSLT 2014 evaluation (2014, methodology): “We further included basic sparse features [34] and we used factors.”
Generalization bounds and consistency for latent-structural probit and ramp loss (2011, background): “Ramp loss updates are closely related to updates derived from n-best lists in training machine translaiton systems [8, 2].”
Learning to Translate with Source and Target Syntax (2010, methodology): “We therefore propose to move these constraints from the formalism into the model, implemented as features in the hierarchical phrase-based model Hiero (Chiang, 2005).”