Findings of the 2012 Workshop on Statistical Machine Translation

By Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, Lucia Specia. 2012

377 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Findings of the 2016 Conference on Machine Translation (2016, methodology): “The fifth edition of the WMT shared task on quality estimation (QE) of machine translation (MT) builds on the previous editions of the task (Callison-Burch et al., 2012; Bojar et al., 2013, 2014, 2015), with “traditional” tasks at sentence and word levels, a new task for entire documents quality pre”
Findings of the 2015 Workshop on Statistical Machine Translation (2015, neutral): “The fourth edition of the WMT shared task on quality estimation (QE) of machine translation (MT) builds on the previous editions of the task (Callison-Burch et al., 2012; Bojar et al., 2013, 2014), with tasks including both sentence and word-level estimation, using new training and test datasets, an”
Findings of the 2014 Workshop on Statistical Machine Translation (2014, methodology): “For all sentence-level these tasks, the same 17 features as in WMT12-13 were used to build base-line systems.”
chrF: character n-gram F-score for automatic MT evaluation (2015, methodology): “System-level correlations The evaluation metrics were compared with human rankings on the system-level by means of Spearman’s correlation coefficients ρ for the WMT12 and WMT13 data and Pearson’s correlation coefficients r for the WMT14 data.”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, neutral): “This conference builds on twelve previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a, 2017).”
Findings of the 2017 Conference on Machine Translation (WMT17) (2017, methodology): “This conference builds on eleven previous editions of WMT as workshops and conference (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a).”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, neutral): “This conference builds on 13 previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016, 2017, 2018).”
Chinese Poetry Generation with Recurrent Neural Networks (2014, background): “Aside from being an approximation of human judgment (Callison-Burch et al., 2012), BLEU might be unnecessarily conservative for poem composition which by its very nature is a creative endeavor.”
Fitting Sentence Level Translation Evaluation with Many Dense Features (2014, methodology): “We use human judgments from the WMT tasks: WMT13 is used for training whereas WMT12 for testing.”
BEER: BEtter Evaluation as Ranking (2014, methodology): “Tuned metric is tested on the human evaluated data from WMT12 (Callison-Burch et al., 2012) for correlation with the human judgment.”
A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU (2014, background): “Many other metrics correlate better with human judgments of translation quality than BLEU, as shown in recent WMT Evaluation Task reports (Callison-Burch et al., 2011; Callison-Burch et al., 2012).”
Sheffield Systems for the English-Romanian WMT Translation Task (2016, methodology): “In previous WMT editions (Callison-Burch et al., 2012; Macháček and Bojar, 2013), SIMPBLEU has been shown to have better correlation than BLEU for the assessment of translation quality at sentence-level.”
Adaptation of machine translation for multilingual information retrieval in the medical domain (2014, methodology): “For the general domain (denoted as gen), we use the test sets provided for the WMT translation task 2012 and 2013 [14] as development and test sets, respectively.”
A Systematic Exploration of Diversity in Machine Translation (2013, background): “Estimation of the quality of the output (“confidence estimation”) has recently gained interest in the MT community (Specia et al., 2011; Bach et al., 2011; Callison-Burch et al., 2012; Bojar et al., 2013), including specifically for post-editing (Tatsumi, 2009; Specia, 2011; Koponen, 2012).”
Exploiting Qualitative Information from Automatic Word Alignment for Cross-lingual NLP Tasks (2013, methodology): “As an example, most supervised approaches to MT quality estimation (Blatz et al., 2003; Callison-Burch et al., 2012) and CLTE (Wäschle and Fendrich, 2012) include features that consider the amount of equivalent terms that are found in the input sentence pairs.”
Online Multitask Learning for Machine Translation Quality Estimation (2015, methodology): “narios such as those proposed by the shared tasks organized within the WMT workshop on SMT (Callison-Burch et al., 2012; Bojar et al., 2013; Bojar et al., 2014).”
Design and Analysis of a Large Corpus of Post-Edited Translations: Quality Estimation, Failure Analysis and the Variability of Post-Edition (2013, methodology): “In the experiments described in this section, each translation is represented by 336 numerical features, most of which are inspired by works in QE for MT (Callison-Burch et al., 2012).”
Modelling Annotator Bias with Multi-task Gaussian Processes: An Application to Machine Translation Quality Estimation (2013, methodology): “For an overview of various algorithms and features we refer the reader to the WMT12 shared task on QE (Callison-Burch et al., 2012).”