The JHU Parallel Corpus Filtering Systems for WMT 2018

By Huda Khayrallah, Hainan Xu, Philipp Koehn. 2018

8 citations recorded by the source corpus.

This work describes our submission to the WMT18 Parallel Corpus Filtering shared task. We use a slightly modified version of the Zipporah Corpus Filtering toolkit (Xu and Koehn, 2017), which computes an adequacy score and a fluency score on a sentence pair, and use a weighted sum of the scores as the selection criteria. This work differs from Zipporah in that we experiment with using the noisy corpus to be filtered to compute the combination weights, and thus avoids generating synthetic data as in standard Zipporah.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Cross-lingual Retrieval for Iterative Self-Supervised Training (2020, background): “In the supervised machine learning literature, data augmentation [5, 26, 50, 59] has been applied to improve learning performance.”
Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering (2018, background): “, 2018) U Tartu University of Tartu, Estonia (Barbu and Barbu Mititelu, 2018) JHU Johns Hopkins University, USA (Khayrallah et al., 2018) LMU Ludwig Maximilian University of Munich, Germany (Hangya and Fraser, 2018) MAJE WebInterpret, Spain (Fomicheva and González-Rubio, 2018) Microsoft Microsoft Co”
Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings (2019, methodology): “Zipporah (Xu and Koehn, 2017; Khayrallah et al., 2018), which is often used as a baseline comparison, uses language model and word translation scores, with weights optimized to separate”