UM-Corpus: A Large English-Chinese Parallel Corpus for Statistical Machine Translation

By Liang Tian, Derek F. Wong, Lidia S. Chao, Paulo Quaresma, Francisco Oliveira, Lu Yi. 2014

135 citations recorded by the source corpus.

Parallel corpus is a valuable resource for cross-language information retrieval and data-driven natural language processing systems, especially for Statistical Machine Translation (SMT). However, most existing parallel corpora to Chinese are subject to in-house use, while others are domain specific and limited in size. To a certain degree, this limits the SMT research. This paper describes the acquisition of a large scale and high quality parallel corpora for English and Chinese. The corpora constructed in this paper contain about 15 million English-Chinese (E-C) parallel sentences, and more than 2 million training data and 5,000 testing sentences are made publicly available. Different from previous work, the corpus is designed to embrace eight different domains. Some of them are further categorized into different topics. The corpus will be released to the research community, which is available at the NLP2CT website.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Feng
2. Xipeng Qiu
3. Shuhao Gu
4. Yang Liu
5. Qun Liu
6. Lei Li
7. Mingxuan Wang
8. Wanying Xie

Citations in the corpus

• Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation (2020, methodology): “We choose the parallel sentences with the domain label Laws from the UM-Corpus (Tian et al., 2014) as our in-domain data.”
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021, methodology): “We choose the parallel sentences with the domain label Thesis from the UM-Corpus (Tian et al., 2014) as our indomain data.”
• UM-pAligner: Neural Network-Based Parallel Sentence Identification Model (2018, background): “Automatic construction of parallel corpora has been an important and active research direction in the NLP community (Tian et al., 2014; Chao et al., 2018; Neves, 2017).”
• Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination (2018, methodology): “To ensure data quality, we chose only the parallel sentences with domain label Laws , Spoken , and Thesis from UM-Corpus, and the LDC bilingual sentences related to News domain as our dataset.”
• Finding Sparse Structures for Domain Specific Neural Machine Translation (2021, methodology): “We selected 6 target domain datasets from from UM-Corpus (Tian et al. 2014).”
• Improving Domain Adaptation Translation with Domain Invariant and Specific Information (2019, methodology): “We chose 109K, 1K and 1K sentences from the UM-Corpus randomly as our training, development and test data.”