Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering

By Philipp Koehn, Huda Khayrallah, Kenneth Heafield, Mikel L. Forcada. 2018

121 citations recorded by the source corpus.

We posed the shared task of assigning sentence-level quality scores for a very noisy corpus of sentence pairs crawled from the web, with the goal of sub-selecting 1% and 10% of high-quality data to be used to train machine translation systems. Seventeen participants from companies, national research labs, and universities participated in this task.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, background): “For example, previous work has explored how to best filter noisy datasets (Koehn et al., 2018, 2019).”
Neural Machine Translation for English–Kazakh with Morphological Segmentation and Synthetic Data (2019, background): “Each ﬁltering step improves the BLEU score, corroborating previous research, e.g. (Koehn et al., 2018), that has shown that noisy sentence pairs indeed cause a drop in translation performance.”
Parallel Corpus Filtering via Pre-trained Language Models (2020, methodology): “We use the WMT 2018 Parallel Corpus Filtering shared task (Koehn et al., 2018) as a benchmark to evaluate our methods.”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, background): “, 2018), and • parallel corpus filtering (Koehn et al., 2018b).”
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2019, background): “Using multilingual noisy webcrawls such as ParaCrawl3 for filtering good quality sentence pairs has been explored in the shared tasks for high resource (Koehn et al., 2018) and low resource (Koehn et al.”
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, background): “…out noise in parallel data but most recently this topic has gained a lot of momentum, partly due to the lack of robustness of neural models and fostered by recent shared tasks on parallel corpus ﬁltering under high-resource (Koehn et al., 2018) and low-resource data conditions (Koehn et al., 2019).”
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2019, background): “Using multilingual noisy web-crawls such as ParaCrawl 2 for ﬁltering good quality sentence pairs has been explored in the shared tasks for high resource (Koehn et al., 2018) and low resource (Koehn et al., 2019) languages.”
A Massive Collection of Cross-Lingual Web-Document Pairs (2019, methodology): “For instance, in the WMT Parallel Corpus Filtering tasks (Koehn et al., 2018, 2019) the downstream performance on a translation task is used as a proxy to determine the quality of a similarity (or filtering) function.”
Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings (2019, methodology): “Recently, there is an increased interest in the filtering of noisy parallel corpora (such as Paracrawl1) to increase the amount of data that can be used to train translation systems (Koehn et al., 2018).”
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases (2022, unknown): “…work on parallel dataset mining have been used mostly in machine translation using document retrieval (Munteanu and Marcu, 2005), language models (Koehn et al., 2018; Koehn et al., 2019), and embedding space alignment (Artetxe and Schwenk, 2019b) to create large corpora (Tiedemann, 2012…”
An Unsupervised System for Parallel Corpus Filtering (2018, methodology): “eu sentence pairs for training both statistical and neural MT systems (Koehn et al., 2018).”
Neural Machine Translation: A Review (2019, methodology): “The most effective approaches in the WMT18 shared task on corpus filtering for NMT (Koehn et al., 2018) used a combination of likelihood scores”
Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora (2018, methodology): “We evaluated our method in the context of the WMT2018 Shared Task on Parallel Corpus Filtering (Koehn et al., 2018) and submitted our best method to the task.”
Microsoft’s Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data (2018, methodology): “The scoring method introduced in this section is our main contribution to the WMT2018 Shared Task on Parallel Corpus Filtering (Koehn et al., 2018), details are provided in our corresponding system submission (Junczys-Dowmunt, 2018).”
Parallel Sentence Mining by Constrained Decoding (2020, methodology): “based on this (Junczys-Dowmunt, 2018) won the WMT 2018 shared task on parallel corpus filtering (Koehn et al., 2018).”
Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions (2019, methodology): “We report on the findings of the WMT 2019 Shared Task on Parallel Corpus Filtering.”
Measuring sentence parallelism using Mahalanobis distances: The NRC unsupervised submissions to the WMT18 Parallel Corpus Filtering shared task (2018, background): “…parallel corpora including past WMT data, Europarl (Koehn, 2005), etc., and 2. a large, potentially “dirty” corpus (i.e., one that may contain non-parallel data, nonlinguistic data, etc.) scraped from the internet (Koehn et al., 2018a), can one identify which sentences from (2) are clean?”