An Efficient Method for Determining Bilingual Word Classes

By Franz Joseph Och. 1999

In statistical natural language processing we always face the problem of sparse data. One way to reduce this problem is to group words into equivalence classes which is a standard method in statistical language modeling. In this paper we describe a method to determine bilingual word classes suitable for statistical machine translation. We develop an optimization criterion based on a maximum-likelihood approach and describe a clustering algorithm. We will show that the usage of the bilingual word classes we get can improve statistical machine translation.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Edinburgh’s Phrase-based Machine Translation Systems for WMT-14 (2014, methodology): “We computed the clusters with GIZA++’s mkcls (Och, 1999) on the source and target side of the parallel training corpus.”
Massively Multilingual Word Embeddings (2016, background): “Word clusters are a related form of distributional representation; in clustering, cross-lingual distributional representations were proposed as well (Och, 1999; T¨ackstr¨om et al., 2012).”
EU-BRIDGE MT: Combined Machine Translation (2014, methodology): “Both systems utilize a language model based on automatically learned word classes using the MKCLS algorithm (Och, 1999).”
The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015 (2015, methodology): “We generated Och clusters (Och, 1999) — a variant of Brown clusters — using mkcls.”
The QT21/HimL Combined Machine Translation System (2016, methodology): “Two of them are automatic word class-based (Och, 1999) language models, using 100 and 1,000 word classes.”
Pre-Translation for Neural Machine Translation (2016, methodology): “In the baseline system, we use three language models, a word-based, a bilingual (Niehues et al., 2011) and a cluster based language model, using 100 automatically generated clusters using MKCLS (Och, 1999).”
WordICA—emergence of linguistic representations for words by independent component analysis (2010, background): “Moreover, the POS tags can be relevant in improving the alignment (Och 1999) and evaluation processes (Callison-Burch et al. 2008) related to machine translation.”
English SLT and MT system description for the IWSLT 2013 evaluation (2013, methodology): “We computed the clusters with GIZA++’s mkcls [38] on the target side of the parallel training corpus.”
The Karlsruhe Institute of Technology Translation Systems for the WMT 2014 (2014, methodology): “The word classes are automatically learned by clustering the words of the corpus using the MKCLS algorithm (Och, 1999).”
Investigating the Usefulness of Generalized Word Representations in SMT (2014, methodology): “We obtained Och clusters using the mkcls utility (Och, 1999) in GIZA++ (Och and Ney, 2003).”
The Karlsruhe Institute of Technology Translation Systems for the WMT 2015 (2015, methodology): “In addition, we use language models based on word classes learned by clustering the words of the corpus using the MK-CLS algorithm (Och, 1999).”
The JHU Machine Translation Systems for WMT 2016 (2016, methodology): “We generated Och clusters (Och, 1999) — a variant of Brown clusters — using mkcls.”
The RWTH Aachen German-English Machine Translation System for WMT 2015 (2015, methodology): “We trained 200 classes on the target side of the bilingual training data (Brown et al., 1992; Och, 1999).”
A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation (2007, methodology): “This is the shared task baseline system for the 2006 NAACL/HLT workshop on statistical machine translation (Koehn and Monz, 2006) and consists of the Pharaoh decoder (Koehn, 2004), SRILM (Stol-cke, 2002), GIZA++ (Och and Ney, 2003), mkcls (Och, 1999), Carmel, 1 and a phrase model training code.”
The KIT-LIMSI Translation System for WMT 2014 (2014, methodology): “The system further applies a language model based on 1000 automatically learned word classes using the MKCLS algorithm (Och, 1999).”
NRC Russian-English Machine Translation System for WMT 2016 (2016, unknown): “Word classes are built using mkcls (Och, 1999).”