Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

By Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark A. Przybocki, Omar Zaidan. 2010

233 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Findings of the 2016 Conference on Machine Translation (2016, background): “This conference builds on nine previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015).”
Findings of the 2015 Workshop on Statistical Machine Translation (2015, neutral): “The hybrid word alignment method combines different kinds of word alignment: GIZA++ word alignment with the grow-diag-ﬁnal-and (GDFA) heuristic (Koehn, 2010), SymGiza++ (Junczys-Dowmunt and Szal, 2011), the Berkeley aligner (Liang et al., 2006), and the edit distance-based aligners (Snover et al.,…”
Findings of the 2014 Workshop on Statistical Machine Translation (2014, neutral): “This workshop builds on eight previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013).”
Neural Architectures for Named Entity Recognition (2016, methodology): “…networks using the back-propagation algorithm updating our parameters on every training example, one at a time, using stochastic gradient descent (SGD) with 2(Graff, 2011; Biemann et al., 2007; Callison-Burch et al., 2010; Parker et al., 2009) a learning rate of 0.01 and a gradient clipping of 5.0.”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, neutral): “This conference builds on twelve previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a, 2017).”
Findings of the 2017 Conference on Machine Translation (WMT17) (2017, methodology): “This conference builds on eleven previous editions of WMT as workshops and conference (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a).”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, neutral): “This conference builds on 13 previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016, 2017, 2018).”
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning (2020, background): “…tasks is a natural and realistic paradigm, and one which shares natural similarities to the evaluation challenges found in various natural language generation tasks such as summarization (Radev et al., 2003) and translation (Callison-Burch et al., 2010) and should be an area of active research.”
Results of the WMT16 Tuning Shared Task (2016, result): “BLEU (Papineni et al., 2002) was shown to be very hard to surpass (Cer et al., 2010) as a tuning metric and this is also confirmed by the previous WMT15 Tuning Task results (Stanojević et al., 2015) and by the results of the invitation-only WMT11 Tunable Metrics Task (Callison-Burch et al., 2010)1.”
Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation (2010, methodology): “Other output formats are ‘-o smatrix’ (one metric per line, each system in a different column) and ‘o nist’ which saves metric scores into files complying with the NIST output format as specified in the Metrics MaTr Evaluation Plan.”
PET: a Tool for Post-editing and Assessing Machine Translation (2012, methodology): “Two datasets were collected using news source sentences from development and test sets provided by WMT8 (Callison-Burch et al., 2010), translated using a phrase-based SMT system built using Moses9 (Koehn et al.”
Findings of the 2011 Workshop on Statistical Machine Translation (2011, methodology): “Our evaluation shared task is similar to the MetricsMATR workshop (Metrics for MAchine TRanslation) that NIST runs (Przy-bocki et al., 2008; Callison-Burch et al., 2010).”
Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems (2011, methodology): “The Meteor1 metric (Banerjee and Lavie, 2005; Denkowski and Lavie, 2010b) has been shown to have high correlation with human judgments in evaluations such as the 2010 ACL Workshop on Statistical Machine Translation and NIST Metrics MATR (Callison-Burch et al., 2010).”
Findings of the 2013 Workshop on Statistical Machine Translation (2013, unknown): “This workshop builds on seven previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012).”
Continuous Measurement Scales in Human Evaluation of Machine Translation (2013, methodology): “Amazon allow location restrictions, based on the registered residential address of the Turker, which can be used to select in favor of those likely to have at least some level of ﬂuency (Callison-Burch et al., 2010).”
Dirt Cheap Web-Scale Parallel Text from the Common Crawl (2013, methodology): “For comparison, we also test on the WMT 2010 test set (Callison-Burch et al., 2010).”