HMM-Based Word Alignment in Statistical Translation

By Stephan Vogel, Hermann Ney, Christoph Tillmann. 1996

In this paper, we describe a new model for word alignment in statistical translation and present experimental results. The idea of the model is to make the alignment probabilities dependent on the differences in the alignment positions rather than on the absolute positions. To achieve this goal, the approach uses a first-order Hidden Markov model (HMM) for the word alignment problem as they are used successfully in speech recognition for the time alignment problem. The difference to the time alignment HMM is that there is no monotony constraint for the possible word orderings. We describe the details of the model and test the model on several bilingual corpora.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation (2020, methodology): “For each target Spanish pronoun, we use an implementation of Vogel et al. (1996) to ﬁnd the corresponding English pronoun and count the translation as correct if the gender tag on the Spanish agrees with the English pronoun gender.”
Learning Semantic Correspondences with Less Supervision (2009, methodology): “The alignment aspect of our model is similar to the HMM model for word alignment (Ney and Vogel, 1996).”
Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources (2004, methodology): “1993) and the HMM alignment (Vogel et al. 1996), along with various improvements and modifications motivated by experimentation by Och & Ney (2000).”
UdS-Sant: English–German Hybrid Machine Translation System (2015, background): “In another well-known word alignment approach, Hidden Markov Model (HMM: (Vogel et al., 1996)), the alignment probabilities depend on the alignment position of the previous word.”
Word Alignment by Fine-tuning Embeddings on Parallel Corpora (2021, methodology): “Based on the IBM translation models (Brown et al., 1993), many statistical word aligners have been proposed (Vogel et al., 1996; Östling and Tiedemann, 2016), including the current most popular tools GIZA++ (Och and Ney, 2000, 2003; Gao and Vogel, 2008) and fast align (Dyer et al., 2013).”
Jointly Learning to Align and Translate with Transformer Models (2019, methodology): “, 1993) and (Vogel et al., 1996), respectively.”
A Discriminative Matching Approach to Word Alignment (2005, methodology): “One source of constraint which our model still does not explicitly capture is the ﬁrst-order dependency between alignment positions, as in the HMM model (Vogel et al., 1996) and IBM models 4+.”
Latent Alignment and Variational Attention (2018, methodology): “Related Work Latent alignment has long been a core problem in NLP, starting with the seminal IBM models [11], HMM-based alignment models [75], and a fast log-linear reparameterization of the IBM 2 model [20].”
An Efficient Method for Determining Bilingual Word Classes (1999, methodology): “The alignment a J that we use is the Viterbi-Alignment of an HMM alignment model similar to (Vogel et al., 1996 counts how often the words in class F are aligned to words in class E.”
Statistical machine translation (2010, unknown): “One glaring omission from this chapter is a discussion of the alignment HMM (Vogel, Ney, and Tillmann 1996).”
Parallel Implementations of Word Alignment Tool (2008, methodology): “The word alignment models implemented in GIZA++, the so-called IBM (Brown et al., 1993) and HMM alignment models (Vogel et al., 1996) are typical implementation of the EM algorithm (Dempster et al., 1977).”
An Attentional Model for Speech Translation Without Transcription (2016, background): “…is overly simplistic, in that it is incapable of capturing patterns in the attention over different positions i. Recognising and exploiting these kinds of patterns has proven critical in traditional word based models of translation (Brown et al., 1993; Vogel et al., 1996; Dyer et al., 2013).”
UM-Corpus: A Large English-Chinese Parallel Corpus for Statistical Machine Translation (2014, methodology): “The word alignment was trained with ten iterations of IBM model 1 and model 4 (Brown et al., 1990; 1993) and six iterations of the HMM alignment model (Vogel et al., 1996).”
Asymmetric Features Of Human Generated Translation (2014, methodology): “The HMM alignment model (Vogel et al., 1996) trained on WMT data was used to word-align the Hansard corpus while replacing words with their corresponding POS tags.”