Zipporah: a Fast and Scalable Data Cleaning System for Noisy Web-Crawled Parallel Corpora

By Hainan Xu, Philipp Koehn. 2017

84 citations recorded by the source corpus.

We introduce Zipporah, a fast and scalable data cleaning system. We propose a novel type of bag-of-words translation feature, and train logistic regression models to classify good data and synthetic noisy data in the proposed feature space. The trained model is used to score parallel sentences in the data pool for selection. As shown in experiments, Zipporah selects a high-quality parallel corpus from a large, mixed quality data pool. In particular, for one noisy dataset, Zipporah achieves a 2.1 BLEU score improvement with using 1/5 of the data over using the entire corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

Citations in the corpus

Scaling Neural Machine Translation (2018, methodology): “Previous work on Paracrawl considered training only on filtered subsets of less than 30M pairs (Xu and Koehn, 2017).”
Cross-lingual Retrieval for Iterative Self-Supervised Training (2020, background): “In the supervised machine learning literature, data augmentation [39, 14, 1, 31] has been applied to improve learning performance.”
Parallel Corpus Filtering via Pre-trained Language Models (2020, methodology): “Xu and Koehn (2017) did similar work to train a ﬁl-tering classiﬁer on synthetic data, but via bag-of-words translation features.”
Unsupervised Question Decomposition for Question Answering (2020, methodology): “Thus, we take inspiration from work in machine translation on parallel corpus mining (Xu & Koehn, 2017; Artetxe & Schwenk, 2019) and in unsupervised QA (Lewis et al., 2019).”
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2021, neutral): “, 2020) and downstream task performance (Moore and Lewis, 2010; Rarrick et al., 2011; Xu and Koehn, 2017; Khayrallah and Koehn 2018; Brown et al., 2020).”
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (2018, background): “For instance, Zipporah learns a classiﬁer over bag-of-word features to distinguish between ground truth translations and synthetic noisy ones (Xu and Koehn, 2017).”
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, methodology): “Participants used a variety of such scoring functions, including n -gram or neural language models on clean data (Rossenbach et al., 2018), language models trained on the provided raw data as contrast, neural translation models (Junczys-Dowmunt, 2018), bag-of-words lexical translation probabilities ”
Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering (2018, methodology): “For example: Taghipour et al. (2011) use an outlier detection algorithm to ﬁlter a parallel corpus; Xu and Koehn (2017) generate synthetic noisy data (inadequate and non-ﬂuent translations) and use this data to train a clas-siﬁer to identify good sentence pairs from a noisy corpus; and Cui et al.…”
Two New Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English (2019, methodology): “Filtering at random gives a BLEU score close to 0, while applying the Zipporah filtering method provides an improvement over using the unfiltered Paracrawl directly, +0.5 BLEU for Nepali–English and +6.0 BLEU for Sinhala– English .”
On the Impact of Various Types of Noise on Neural Machine Translation (2018, methodology): “For example: Taghipour et al. (2011) use an outlier detection algorithm to ﬁlter a parallel corpus; Xu and Koehn (2017) generate synthetic noisy data (inadequate and non-ﬂuent translations) and use this data to train a clas-siﬁer to identify good sentence pairs from a noisy corpus; and Cui et al.…”
Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings (2019, methodology): “Zipporah (Xu and Koehn, 2017; Khayrallah et al., 2018), which is often used as a baseline comparison, uses language model and word translation scores, with weights optimized to separate clean and synthetic noise data.”
Effective Parallel Corpus Mining using Bilingual Sentence Embeddings (2018, methodology): “Finally, our method has a modest degree of correlation with the pair quality scores provided by Zipporah (Xu and Koehn, 2017).”
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples (2023, background): “Some are concerned with detecting and filtering web datasets (Dodge et al., 2021; Xu & Koehn, 2017), employing traditional detection techniques such as n-gram overlap.”
Curriculum Learning for Domain Adaptation in Neural Machine Translation (2019, methodology): “4 We filter the data using the Zipporah cleaning tool (Xu and Koehn, 2017), with a threshold score of 1.”
The JHU Parallel Corpus Filtering Systems for WMT 2018 (2018, methodology): “We use a slightly modiﬁed version of the Zipporah Corpus Filtering toolkit (Xu and Koehn, 2017).”
Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation (2019, methodology): “Furthermore, our method achieved the highest precision, out-performing the baseline in all three language pairs, which is more important when mining from the web (Xu and Koehn, 2017).”