Findings of the 2011 Workshop on Statistical Machine Translation

By Chris Callison-Burch, Philipp Koehn, Christof Monz, Omar Zaidan. 2011

255 citations recorded by the source corpus.

Subfields: Natural Language Processing, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Machine Translation

Citations in the corpus

Findings of the 2016 Conference on Machine Translation (2016, background): “This conference builds on nine previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015).”
Findings of the 2015 Workshop on Statistical Machine Translation (2015, neutral): “This workshop builds on eight previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014).”
Findings of the 2014 Workshop on Statistical Machine Translation (2014, neutral): “The agreement rates are more or less in line with prior years: worse for some tasks, better for others, and on average, the best since WMT11 (where agreement scores were likely inﬂated due to inclusion of reference translations in the comparisons).”
Meteor Universal: Language Specific Translation Evaluation for Any Target Language (2014, background): “…have seen a variety of metrics employ language speciﬁc resources to replicate human translation rankings far better than simple baselines (Callison-Burch et al., 2011; Callison-Burch et al., 2012; Mach´aˇcek and Bojar, 2013; Snover et al., 2009; Denkowski and Lavie, 2011; Dahlmeier et…”
Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses (2017, methodology): “Related to our approach is the literature on novel methods for the evaluation of machine translation systems, especially through the WMT evaluation task (Callison-Burch et al., 2011; Machácek and Bojar, 2014; Stanojevic et al., 2015).”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, neutral): “This conference builds on twelve previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a, 2017).”
Findings of the 2017 Conference on Machine Translation (WMT17) (2017, methodology): “This conference builds on eleven previous editions of WMT as workshops and conference (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a).”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, neutral): “This conference builds on 13 previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016, 2017, 2018).”
On Evaluating and Comparing Conversational Agents (2018, background): “There has also been work on learning scoring models for evaluation of MT models in WMT evaluation task (Callison-Burch et al., 2011; Bojar et al., 2016).”
A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU (2014, background): “Many other metrics correlate better with human judgments of translation quality than BLEU, as shown in recent WMT Evaluation Task reports (Callison-Burch et al., 2011; Callison-Burch et al., 2012).”
Automatically Assessing Machine Summary Content Without a Gold Standard (2013, methodology): “Shared tasks on system combination have also been organized in recent years to encourage the development of such methods (Callison-Burch et al. 2010, 2011).”
Beyond BLEU:Training Neural Machine Translation with Semantic Similarity (2019, methodology): “The WMT shared tasks included tunable metric tasks in 2011 (Callison-Burch et al., 2011) and again in 2015 (Stanojevi´c et al., 2015) and 2016 (Jawaid et al., 2016).”
Large Language Models Are State-of-the-Art Evaluators of Translation Quality (2023, methodology): “Initially, Callison-Burch et al. (2011) proposed to use Kendall’s Tau-a ignoring human rankings that tied, while penalising ties in automatic metrics. where Concordant is the set of all human segment comparisons for which a given metric suggests the same order of systems and Discordant is the set…”
Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation (2015, background): “the tested alternatives could consistently outperform BLEU (Cer et al., 2010; Callison-Burch et al., 2011).”
Referential Translation Machines for Quality Estimation (2013, methodology): “Both of these models learn a regression function using the features to estimate a numerical target value such as the HTER score, the F 1 score (Bic¸ici and Yuret, 2011b), or the PET score.”
Continuous Measurement Scales in Human Evaluation of Machine Translation (2013, background): “…was structured as ranking translations from best-to-worst, or by direct estimation of ﬂuency and adequacy – of agreement between annotators decreasing (Callison-Burch et al., 2008; Callison-Burch et al., 2009; Callison-Burch et al., 2010; Callison-Burch et al., 2011; Callison-Burch et al., 2012).”
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context (2017, methodology): “For En-Fr, we use the ﬁrst 10M lines from the Giga-EnFr corpus released as part of the WMT shared task (Callison-Burch et al., 2011).”
Putting Human Assessments of Machine Translation Systems in Order (2012, methodology): “(Callison-Burch et al., 2011) The workshop’s human evaluation component has been gradually reﬁned over several years, and as a consequence it has produced a fantastic collection of publicly available data consisting primarily of pair-wise judgements of translation systems made by human assessors…”