An End-to-End Discriminative Approach to Machine Translation

By Percy Liang, Alexandre Bouchard-Côté, Daniel Klein, B. Taskar. 2006

310 citations recorded by the source corpus.

We present a perceptron-style discriminative approach to machine translation in which large feature sets can be exploited. Unlike discriminative reranking approaches, our system can take advantage of learned features in all stages of decoding. We first discuss several challenges to error-driven discriminative approaches. In particular, we explore different ways of updating parameters given a training example. We find that making frequent but smaller updates is preferable to making fewer but larger updates. Then, we discuss an array of features and show both how they quantitatively increase BLEU score and how they qualitatively interact on specific examples. One particular feature we investigate is a novel way to introduce learning into the initial phrase extraction process, which has previously been entirely heuristic.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Sequence-Level Knowledge Distillation (2016, methodology): “This approach is inspired by local updating (Liang et al., 2006) and hope / fear training (Chiang, 2012), which are commonly-used methods for discriminative training in statistical machine translation (although to our knowledge not for knowledge distillation).”
Learning Continuous Phrase Representations for Translation Modeling (2014, methodology): “Following Liang et al. (2006), we assume that every translation candidate is always coupled with a corresponding 𝐴 , called the Viterbi derivation , generated by (1).”
Mixture Models for Diverse Machine Translation: Tricks of the Trade (2019, methodology): “(3)Sentence-level BLEU is computed with smoothed n-gram counts (+1) for n > 1 (Liang et al., 2006; Nakov et al., 2012; Ott et al., 2018a).”
11,001 New Features for Statistical Machine Translation (2009, background): “Others have introduced alternative discriminative training meth-ods (Tillmann and Zhang, 2006; Liang et al., 2006; Turian et al., 2007; Blunsom et al., 2008; Macherey et al., 2008), in which a recurring challenge is scalability: to train many features, we need many train-218 ing examples, and to…”
Speech-Centric Information Processing: An Optimization-Oriented Approach (2013, background): “Then, we outline a general solution, expanded and generalized from recently published work, which overcomes the inconsistency and forms the basis of much of the remaining material in this paper.”
Explaining and Generalizing Back-Translation through Wake-Sleep (2018, methodology): “…these previous papers in that we are interested in a principled interpretation of back-translation as a strategy, independent of the particular parameterization in place; our analysis will hold parameterization of a probabilistic MT model, e.g., Liang et al. (2006) and Blunsom and Osborne (2008).”
A Recursive Recurrent Neural Network for Statistical Machine Translation (2014, background): “Large scale feature training has drawn more at-tentions these years (Liang et al., 2006; Yu et al., 2013).”
The Karlsruhe Institute of Technology Translation Systems for the WMT 2015 (2015, methodology): “In our experiments we used the BLEU+1 score introduced by Liang et al. (2006).”
Efficient grounding of abstract spatial concepts for natural language interaction with robot platforms (2018, background): “[7] impose a discriminative structure to the problem of modeling the joint likelihood between words and groundings.”
End-to-End Speech Recognition: A Survey (2023, background): “For these all-neural approaches recently the term end-to-end (E2E) [16], [17], [13], [18] has been established.”
Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations (2011, methodology): “Instead, we make two approximations, described below, leading to a Perceptron-style additive (Collins, 2002) parameter update scheme which has been modified to reason about hidden variables, similar in style to the approaches of (Liang et al., 2006; Zettlemoyer and Collins, 2007), but adapted for ou”
Batch Tuning Strategies for Statistical Machine Translation (2012, methodology): “Inspired by the local oracle of Liang et al. (2006), we deﬁne ˜ E i ∗ to be an oracle set: ˜ E i ∗ = { e | BLEU i ( e ) is maximal } .”
Sequence Level Training with Recurrent Neural Networks (2015, unknown): “BLEU is essentially a geometric mean over n-gram precision scores as well as a brevity penalty (Liang et al., 2006); in this work, we consider up to 4 -grams.”
Statistical learning for semantic parsing: A survey (2019, methodology): “To make the model more powerful, we can add latent variables into the log-linear model ( p.y j x/ P h p.y; h j x/ ) [31,43] .”
Structured Models for Fine-to-Coarse Sentiment Analysis (2007, methodology): “, 2004) and for inference based learning algorithms like those used in this work (Liang et al., 2006).”
Generalization bounds and consistency for latent-structural probit and ramp loss (2011, background): “Ramp loss updates are closely related to updates derived from n-best lists in training machine translaiton systems [8, 2].”
Probabilistic Inference for Machine Translation (2008, background): “Most work on discriminative training for SMT has focussed on linear models, often with margin based algorithms (Liang et al., 2006; Watanabe et al., 2006), or rescaling a product of sub-models (Och, 2003; Ittycheriah and Roukos, 2007).”
Learning Semantic Representations for the Phrase Translation Model (2013, methodology): “We assume that every translation candidate is always coupled with a corresponding 𝐴, called Viterbi derivation, generated by (1).”