Collection of a Large Database of French-English SMT Output Corrections

By Marion Potet, Emmanuelle Esperança-Rodier, Laurent Besacier, H. Blanchon. 2012

52 citations recorded by the source corpus.

Corpus-based approaches to machine translation (MT) rely on the availability of parallel corpora. To produce user-acceptable translation outputs, such systems need high quality data to be efficiency trained, optimized and evaluated. However, building high quality dataset is a relatively expensive task. In this paper, we describe the data collection and analysis of a large database of 10.881 SMT translation output hypotheses manually corrected. These post-editions were collected using Amazon's Mechanical Turk, following some ethical guidelines. A complete analysis of the collected data pointed out a high quality of the corrections with more than 87 % of the collected post-editions that improve hypotheses and more than 94 % of the crowdsourced post-editions which are at least of professional quality. We also post-edited 1,500 gold-standard reference translations (of bilingual parallel corpora generated by professional) and noticed that 72 % of these translations needed to be corrected during post-edition. We computed a proximity measure between the differents kind of translations and pointed out that reference translations are as far from the hypotheses than from the corrected hypotheses (i.e. the post-editions). In light of these last findings, we discuss the adequation of text-based generated reference translations to train setence-to-sentence based SMT systems.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Findings of the 2015 Workshop on Statistical Machine Translation (2015, neutral): “Interest-ingly, and similar to the ﬁndings of Potet et al. (2012), crowdsourced post-editions feature a distance from the professional ones that is even higher than the distance between the original target sentences and the experts’ corrections ( 29 .”
Searching for Context: a Study on Document-Level Labels for Translation Quality Estimation (2015, methodology): “In order to better understand this problem, we conducted an experiment with French-English translations from the LIG corpus (Potet et al., 2012).”
Design and Analysis of a Large Corpus of Post-Edited Translations: Quality Estimation, Failure Analysis and the Variability of Post-Edition (2013, background): “With the development of post-edition, more and more datasets of post-edited translations are being collected and distributed (Potet et al., 2012; Callison-Burch et al., 2012).”
Target-Centric Features for Translation Quality Estimation (2014, background): “Post-editing corpora can play this role and have recently become available (Potet et al., 2012).”