Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions

By Philipp Koehn, Francisco (Paco) Guzmán, Vishrav Chaudhary, Juan Pino. 2019

83 citations recorded by the source corpus.

Following the WMT 2018 Shared Task on Parallel Corpus Filtering, we posed the challenge of assigning sentence-level quality scores for very noisy corpora of sentence pairs crawled from the web, with the goal of sub-selecting 2% and 10% of the highest-quality data to be used to train machine translation systems. This year, the task tackled the low resource condition of Nepali-English and Sinhala-English. Eleven participants from companies, national research labs, and universities participated in this task.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “It is surprising to see how far behind the rest of NLG is behind MT in this regard, where filtering and cleaning of scraped data is common practice and shared tasks are being held (Koehn et al., 2019).”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, background): “• parallel corpus filtering (Koehn et al., 2019)”
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2019, methodology): “, 2018) and low resource (Koehn et al., 2019) languages.”
Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges (2019, background): “13 research groups submitted 24 metrics, 10 of which are reference-less “metrics” and constitute submissions to the joint task with WMT19 Quality Estimation Task, “QE as a Metric”.”
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, background): “…out noise in parallel data but most recently this topic has gained a lot of momentum, partly due to the lack of robustness of neural models and fostered by recent shared tasks on parallel corpus ﬁltering under high-resource (Koehn et al., 2018) and low-resource data conditions (Koehn et al., 2019).”
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2019, background): “, 2018) and low resource (Koehn et al., 2019) languages.”
Detecting Hallucinated Content in Conditional Neural Sequence Generation (2020, methodology): “Additionally, we show how to use these token-level hallucination labels in two case studies to improve self-training (Scudder, 1965) and learning from noisy mined bitext (Koehn et al., 2019) in low-resource MT.”
A Massive Collection of Cross-Lingual Web-Document Pairs (2019, methodology): “For instance, in the WMT Parallel Corpus Filtering tasks (Koehn et al., 2018, 2019) the downstream performance on a translation task is used as a proxy to determine the quality of a similarity (or filtering) function.”
Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings (2019, methodology): “For the task of low-resource filtering (Koehn et al., 2019), we are provided with a very noisy 40.”
PMIndia - A Collection of Parallel Corpora of Languages of India (2020, background): “…were released for the shared news translation tasks in English-Hindi and EnglishGujarati at the WMT conference (Bojar et al., 2014; Barrault et al., 2019), as well as for the corpus-filtering task (Guzmán et al., 2019; Koehn et al., 2019), which targeted English–Sinhala and English–Nepali.”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “For example, WMT-2019 [104] shared task had Nepali–Hindi parallel corpus.”
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases (2022, unknown): “…mining have been used mostly in machine translation using document retrieval (Munteanu and Marcu, 2005), language models (Koehn et al., 2018; Koehn et al., 2019), and embedding space alignment (Artetxe and Schwenk, 2019b) to create large corpora (Tiedemann, 2012 (Barzilay and Lee, 2003)…”