Findings of the 2014 Workshop on Statistical Machine Translation

By Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand and 3 more. 2014

850 citations recorded by the source corpus.

This paper presents the results of the WMT14 shared tasks, which included a standard news translation task, a separate medical translation task, a task for run-time estimation of machine translation quality, and a metrics task. This year, 143 machine translation systems from 23 institutions were submitted to the ten translation directions in the standard translation task. An additional 6 anonymized systems were included, and were then evaluated both automatically and manually. The quality estimation task had four subtasks, with a total of 10 teams, submitting 57 entries

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, methodology): “For WMT English to German, we use the same training data as [Vaswani et al., 2017] (i.e. News Commentary v13, Common Crawl, Europarl v7) and newstest2013 as a validation set [Bojar et al., 2014].”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, methodology): “We train machine translation (MT) models on WMT14 English-German (Bojar et al., 2014) and WMT17 English-Russian (Bojar et al., 2017).”
Findings of the 2016 Conference on Machine Translation (2016, methodology): “This conference builds on nine previous WMT workshops (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015).”
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (2021, methodology): “Table 1: We report BLEU scores for variants of the GPT-3 model using different prompt formats on the WMT’14 Fr-En dataset [1] as measured by SacreBLEU [19].”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “Table 5: Four prompts for the WMT’14 dataset (Bojar et al., 2014) for MT evaluation.”
Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, background): “Thus it is more open ended, or, equivalently, it has a higher entropy than say machine translation Bojar et al. (2014) which has few correct translations that are very similar to each other.”
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models (2022, methodology): “For Fr-En and De-En, we take WMT newstest2013 newstest2014 (Bojar et al., 2014) as the development sets and test sets, respectively.”
C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models (2023, background): “, 2018), machine translation (Bojar et al., 2014), and summarization (Hermann et al.”
How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation (2016, background): “…how well automatic metrics correlate with human judgements in in both machine translation (Callison-Burch et al., 2010; Callison-Burch et al., 2011; Bojar et al., 2014; Graham et al., 2015) and natural language generation (NLG) (Stent et al., 2005; Cahill, 2009; Reiter and Belz, 2009; Es-pinosa…”
Do Transformer Modifications Transfer Across Implementations and Applications? (2021, methodology): “For supervised training on the WMT’14 English to German translation task (Bojar et al., 2014), we use the same model and batch size as for the transfer learning setting.”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, neutral): “This conference builds on twelve previous editions of WMT as workshops and conferences (Koehn and Monz, 2006; Callison-Burch et al., 2007, 2008, 2009, 2010, 2011, 2012; Bojar et al., 2013, 2014, 2015, 2016a, 2017).”
Pre-Trained Models: Past, Present and Future (2021, background): “…deep neural networks, massive efforts have been devoted to manually constructing high-quality datasets for AI tasks (Deng et al., 2009; Lin et al., 2014; Bojar et al., 2014), making it possible to learn effective neural models for specific tasks that are superior to conventional non-neural models.”
Adaptively Sparse Transformers (2019, background): “• WMT 2014 English → German ( EN DE , Bojar et al., 2014): 4.5M sentence pairs.”