Improving Machine Translation Performance by Exploiting Non-Parallel Corpora
By Dragos Stefan Munteanu, Daniel Marcu. 2005
476 citations recorded by the source corpus.
We present a novel method for discovering parallel sentences in comparable, non-parallel corpora. We train a maximum entropy classifier that, given a pair of sentences, can reliably determine whether or not they are translations of each other. Using this approach, we extract parallel data from large Chinese, Arabic, and English non-parallel newspaper corpora. We evaluate the quality of the extracted data by showing that it improves the performance of a state-of-the-art statistical machine translation system. We also show that a good-quality MT system can be built from scratch by starting with a very small parallel corpus (100,000 words) and exploiting a large non-parallel corpus. Thus, our method can be applied with great benefit to language pairs for which only scarce resources are available.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Machine Translation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Graham Neubig
- Luke S. Zettlemoyer
- Mohit Bansal
- Jianfeng Gao
- Kyunghyun Cho
- Percy Liang
- Furu Wei
- Noah A. Smith
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Ming-Hsuan Yang
- Andrew Zisserman
- Tao Qin
- Jitendra Malik
In Natural Language Processing
- Holger Schwenk
- Graham Neubig
- Philipp Koehn
- Luke S. Zettlemoyer
- Jianfeng Gao
- Kyunghyun Cho
- Mikel Artetxe
- Furu Wei
In Machine Translation
Citations in the corpus
- Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (2018, background): “Many of these approaches rely on cross-lingual document retrieval (Utiyama and Isahara, 2003; Munteanu and Marcu, 2005, 2006; Abdul-Rauf and Schwenk, 2009) or machine translation (Abdul-Rauf and Schwenk, 2009; Bouamor and Sajjad, 2018).”
- WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2019, neutral): “(Utiyama and Isahara, 2003; Munteanu and Marcu, 2005) or machine translation, e.”
- ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, methodology): “Munteanu and Marcu (2005) use document retrieval and a maximum entropy classifier to identify parallel sentence pairs in a multi-lingual collection of news stories.”
- CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2019, methodology): “In another work, the mining approach of Munteanu and Marcu (2005) was applied to extract large corpora from Wikipedia in sixteen lan-2 guages (Smith et al., 2010).”
- A Massive Collection of Cross-Lingual Web-Document Pairs (2019, methodology): “Some of these methods have focused on publication date and other temporal heuristics to aid in identifying parallel documents (Munteanu and Marcu, 2005, 2006; Udupa et al., 2009; Do et al., 2009; Abdul-Rauf and Schwenk, 2009).”
- Effective Parallel Corpus Mining using Bilingual Sentence Embeddings (2018, methodology): “Some text-based approaches for identifying bitexts rely on methods such as n-gram scoring (Uszkoreit et al., 2010), named entity matching (Do et al., 2009), and cross-language information retrieval (Utiyama and Isahara, 2003; Munteanu and Marcu, 2005).”
- Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021, background): “Finally coarse-grained divergences might result from the process of automatically mining and aligning corpora from monolingual data (Fung and Cheung, 2004; Munteanu and Marcu, 2005), or web-scale parallel text (Smith et al., 2013; El-Kishky et al., 2020; Espl ` a et al., 2019).”
- Large Scale Parallel Document Mining for Machine Translation (2010, methodology): “Even though the presented approach works on a less granular level than the sentence-level approach of Munteanu and Marcu (2005), we compare results on the same input data 2 used by those authors to automatically generate the Table 7: BLEU scores of the degraded Czech to English baseline systems…”
- Crowdsourcing Translation: Professional Quality from Non-Professionals (2011, methodology): “These include harvesting the web for translations or comparable corpora (Resnik and Smith, 2003; Munteanu and Marcu, 2005; Smith et al., 2010; Uszkoreit et al., 2010), improving SMT models so that they are better suited to the low resource setting (Al-Onaizan et al., 2002; Probst et al., 2002; Oard…”
- Extracting Parallel Sentences from Comparable Corpora using Document Level Alignment (2010, methodology): “The related problem of automatic document alignment in news and web corpora has been explored by a number of researchers, including Resnik and Smith (2003), Munteanu and Marcu (2005), Tillmann and Xu (2009), and Tillmann (2009).”
- On the Use of Comparable Corpora to Improve SMT performance (2009, methodology): “Recent work by Munteanu and Marcu (2005) uses a bilingual lexicon to translate some of the words of the source sentence.”
- Wikipedia as an SMT Training Corpus (2013, methodology): “According to Munteanu and Marcu (2005), a binary classifier can be trained to distinguish between parallel sentences and non-parallel sentences using features such as: word alignment log probability, number of aligned/unaligned words, longest sequence of aligned words, etc.”
- A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation (2007, background): “Munteanu and Marcu (2005) extract parallel sentences from large Chinese, Arabic, and English non-parallel newspaper corpora.”
- UM-pAligner: Neural Network-Based Parallel Sentence Identification Model (2018, methodology): “Munteanu and Marcu (2005) aligned articles by considering the publication date of the documents, and employed a maximum entropy classifier for identifying the parallel sentences from the aligned articles.”