Report on the 11th IWSLT evaluation campaign

By Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, Marcello Federico. 2014

423 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Improving Neural Machine Translation Models with Monolingual Data (2015, methodology): “We use data provided for the IWSLT 14 machine translation track (Cettolo et al., 2014), namely the WIT 3 parallel corpus (Cettolo et al., 2012), which consists of TED talks, and the SETimes corpus (Tyers and Alperen, 2010).”
A Convolutional Encoder Model for Neural Machine Translation (2016, methodology): “We evaluate different encoders and ablate architectural choices on a small dataset from the GermanEnglish machine translation track of IWSLT 2014 (Cettolo et al., 2014) with a similar setting to (Ranzato et al., 2015).”
An Actor-Critic Algorithm for Sequence Prediction (2016, methodology): “For the machine translation experiment, we use data from the German-English machine translation track of the IWSLT 2014 evaluation campaign (Cettolo et al., 2014), as used in Ranzato et al.”
Initialization and Regularization of Factorized Neural Layers (2021, methodology): “Improving Transformer Training and Compression: We start with the original Transformer architecture on the IWSLT-14 translation task (Cettolo et al., 2014) but use an SGD-based training routine as a baseline (Gehring et al.”
Depth-Adaptive Transformer (2019, positive): “translation (Cettolo et al., 2014) as well as WMT’14 English-French translation show that we can match the baseline performance at substantially less computation (§4).”
On the Variance of the Adaptive Learning Rate and Beyond (2019, methodology): “We compare them to vanilla Adam with and without warmup on the IWSLT’14 German to English translation dataset (Cettolo et al., 2014).”
A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation (2017, background): “…base-line systems (Oda et al., 2015; Yin and Neubig, 2017), with BLEU scores more than twice those of machine translation between natural languages (Cettolo et al., 2016), a task that we would expect to be no more difﬁcult than code documentation or code generation, especially given the much…”
Sequence-to-Sequence Learning as Beam-Search Optimization (2016, methodology): “(2016), which uses data from the German-to-English portion of the IWSLT 2014 machine translation evaluation campaign (Cettolo et al., 2014).”
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback (2017, methodology): “We use parallel transcriptions of TED talks for these pairs of languages from the machine translation track of the IWSLT 2014 and 2015 (Cettolo et al., 2014, 2015, 2012).”
Classical Structured Prediction Losses for Sequence to Sequence Learning (2017, methodology): “We experiment on the IWSLT’14 German to English (Cettolo et al., 2014) task using a similar setup as Ranzato et al.”
Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization (2018, methodology): “For German→English task, the bilingual corpus is from IWSLT 2014 evaluation campaign (Cettolo et al. 2014), containing about 153k sentence pairs for training, and 7k/6.5k sentence pairs for validation/test.”
Random Feature Attention (2021, background): “• IWSLT14 DE-EN (Cettolo et al., 2014) is based on TED talks.”
Editable Neural Networks (2020, methodology): “We consider the IWSLT 2014 German-English translation task with the standard training/test splits (Cettolo et al. (2015)). The data is preprocessed with Moses Tokenizer (Koehn et al. (2007)) and converted to lowercase. We further apply the Byte-Pair Encoding with 10.000 BPE rules learned jointly fro”
Regularization techniques for fine-tuning in neural machine translation (2017, background): “Neural machine translation (Bahdanau et al., 2015; Sutskever et al., 2014) has established itself as the new state of the art at recent shared translation tasks (Bojar et al., 2016; Cettolo et al., 2016).”
Concealed Data Poisoning Attacks on NLP Models (2021, methodology): “We use a Transformer model trained on IWSLT 2014 (Cettolo et al., 2014) German-English, which contains 160,239 training examples.”
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem (2020, positive): “For en-he we use 185K sentence pairs from the multilingual TED talks corpus (Cettolo et al., 2014).”
Overview of the IWSLT 2017 Evaluation Campaign (2017, methodology): “For each language direction, the output of the selected 9 systems on the HE set was assigned to professional translators to be post-edited (for all the details about data preparation and postediting see [20, 21, 22]).”
Improving Gender Translation Accuracy with Filtered Self-Training (2021, neutral): “Parallel Data: We train en-fr on the WMT14 news task (Bojar et al., 2014), en-it on the IWSLT13 task (Cettolo et al., 2013), en-ru on WMT16 (Bojar et al., 2016) 15 , en-he on IWSLT14 (Cettolo et al., 2014), and en-de on WMT18 (Bojar et al., 2018) 16 .”
Understanding the Difficulty of Training Transformers (2020, methodology): “, 2017) has led to a series of breakthroughs in various domains (Devlin et al., 2019; Velickovic et al., 2018; Huang et al., 2019; Parmar et al., 2018; Ramachandran et al., 2019). Liu et al. (2020a) show that compared to other architectures, removing the warmup phase is more damaging for Transformer”