Statistical Phrase-Based Translation

By Philipp Koehn, Franz Joseph Och, Daniel Marcu. 2003

3,951 citations recorded by the source corpus.

We propose a new phrase-based translation model and decoding algorithm that enables us to evaluate and compare several, previously proposed phrase-based translation models. Within our framework, we carry out a large number of experiments to understand better and explain why phrase-based models out-perform word-based models. Our empirical results, which hold for all examined language pairs, suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translations. Surprisingly, learning phrases longer than three words and learning phrases from high-accuracy word-level alignment models does not have a strong impact on performance. Learning only syntactically motivated phrases degrades the performance of our systems.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016, methodology): “Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using residual connections as well as attention connections from the decoder network to the encoder.”
Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation (2016, background): “Unlike conventional statistical machine translation (SMT) systems (Koehn et al., 2003; Durrani et al., 2014) which consist of multiple separately tuned components, NMT models encode the source sequence into continuous representation space and generate the target sequence in an end-to-end fashon.”
Effective Approaches to Attention-based Neural Machine Translation (2015, methodology): “Lastly, implementing NMT decoders is easy unlike the highly intricate decoders in standard MT (Koehn et al., 2003).”
Neural Machine Translation by Jointly Learning to Align and Translate (2014, methodology): “Unlike the traditional phrase-based translation system (see, e.g., Koehn et al. , 2003) which consists of many small sub-components that are tuned separately, neural machine translation attempts to build and train a single, large neural network that reads a sentence and outputs a correct…”
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (2014, methodology): “In the phrase-based SMT framework introduced in (Koehn et al., 2003) and (Marcu and Wong, 2002), the translation model log p ( e | f ) is factorized into the translation probabilities of matching phrases in the source and target sentences.”
Meta-Learning for Low-Resource Neural Machine Translation (2018, background): “…Bahdanau et al., 2015; Vaswani et al., 2017), it has been noticed that the vanilla NMT often lags behind conventional machine translation systems, such as statistical phrase-based translation systems (PBMT, Koehn et al., 2003), for low-resource language pairs (see, e.g., Koehn and Knowles, 2017).”
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks (2019, methodology): “In feature extraction ( ) the model’s weights are ‘frozen’ and the pretrained representations are used in a downstream model similar to classic feature-based approaches (Koehn et al., 2003).”
Get To The Point: Summarization with Pointer-Generator Networks (2017, methodology): “Originating from Statistical Machine Translation (Koehn, 2009), coverage was adapted for NMT by Tu et al. (2016) and Mi et al. (2016), who both use a GRU to update the coverage vector each step.”
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “PBSMT is short for phrase-based statistical machine translation [39].”
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (2014, neutral): “This is in stark contrast to the conventional phrase-based machine translation system (Koehn et al., 2003).”
Moses: Open Source Toolkit for Statistical Machine Translation (2007, background): “Phrase-based statistical machine translation (Koehn et al. 2003) has emerged as the dominant paradigm in machine translation research.”
Meteor Universal: Language Specific Translation Evaluation for Any Target Language (2014, background): “First, a standard phrase table is learned from the bitext (Koehn et al., 2003).”
On Using Very Large Target Vocabulary for Neural Machine Translation (2014, background): “Neural machine translation has a number of advantages over the existing statistical machine translation system, speciﬁcally, the phrase-based system (Koehn et al., 2003).”
Addressing the Rare Word Problem in Neural Machine Translation (2014, background): “For more than a decade, the standard MT approach [14] has been subject to intensive research and resulted in better systems over time [13, 4, 3, 8].”
Learning to Write with Cooperative Discriminators (2018, background): “statistical machine translation (Koehn et al., 2003; Och, 2003; Watanabe et al., 2007; Chiang et al., 2009) and remain common with neural machine translation, even when an extremely large amount of data is available (Wu et al.”
Neural Approaches to Conversational AI (2018, positive): “onse generation system. This was the idea originally proposed in one of the ﬁrst works on fully data-driven conversational AI (Ritter et al., 2011), which applied a phrase-based translation approach (Koehn et al., 2003) to dialogue datasets extracted from Twitter (Serban et al., 2015). A different E”
On Using Monolingual Corpora in Neural Machine Translation (2015, methodology): “7 Results and Analysis 7.1 Zh-En: OpenMT’15 In addition to NMT-based systems, we also trained a phrase-based as well as hierarchical phrase-based SMT systems (Koehn et al. , 2003; Chi-ang, 2005) with/without re-scoring by an external neural language model (CSLM, Schwenk, 2007b).”
Phrase-Based & Neural Unsupervised Machine Translation (2018, methodology): “In this section, we discuss how to perform unsupervised machine translation using a PhraseBased Statistical Machine Translation (PBSMT) system (Koehn et al., 2003) as the underlying backbone model.”
Does String-Based Neural MT Learn Source Syntax? (2016, background): “Non-neural machine translation systems consist chieﬂy of phrase-based systems (Koehn et al., 2003) and syntax-based systems (Galley et al., 2004; Galley et al., 2006; DeNeefe et al., 2007; Liu et al., 2011; Cowan et al., 2006), the latter of which adds syntactic information to source side…”
Dual Learning for Machine Translation (2016, background): “State-of-the-art machine translation (MT) systems, including both the phrase-based statistical translation approaches [7, 4, 14] and the recently emerged neural networks based translation approaches [1, 6], heavily rely on aligned parallel training corpora.”