ParaCrawl: Web-scale parallel corpora for the languages of the EU

By Miquel Esplà-Gomis, Mikel L. Forcada, Gema Ramírez-Sánchez, Hieu T. Hoang. 2019

106 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, methodology): “Efforts such as Paracrawl (Esplà-Gomis et al., 2019) have mined for data on the web and are freely usable.”
MURAL: Multimodal, Multitask Retrieval Across Languages (2021, methodology): “We create an Ensemble of Open Bilingual Translation (EOBT) Pairs dataset by combining publicly available datasets, including Europarl (Koehn, 2005), Paracrawl (Esplà et al., 2019), Wikimatrix (Schwenk et al.”
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2021, methodology): “, 2020), ParaCrawl (Esplà et al., 2019; Bañón et al., 2020), WikiMatrix (Schwenk et al.”
The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation (2021, background): “, 2020), and ParaCrawl (Esplà et al., 2019), which contain noisy translations.”
Unsupervised Quality Estimation for Neural Machine Translation (2020, methodology): “We used publicly available MT datasets such as Paracrawl (Es-plà et al., 2019) and Europarl (Koehn, 2005).”
Machine Translation into Low-resource Language Varieties (2021, background): “, 2021), and 26M NO sentences from ParaCrawl (Esplà et al., 2019) combined with the WikiMatrix NO sentences to train embeddings.”
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task (2020, unknown): “, 2020), WikiTitles(2), and ParaCrawl (Esplà-Gomis et al., 2019).”
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022, background): “, 2020), the common approach to collect large amounts of raw textual data still relies primarily on crawled web text (Ortiz Suárez et al., 2019; Ortiz Suárez et al., 2020; Xue et al., 2021; El-Kishky et al., 2020; Esplà et al., 2019; Bañón et al., 2020; Gao et al., 2020), and although some of the in”
Building Machine Translation Systems for the Next Thousand Languages (2022, methodology): “These improvements have largely been driven by advances in machine learning and the availability of large-scale web-mined datasets (Resnik & Smith, 2003; Uszkoreit et al., 2010; Esplà-Gomis, 2009; Esplà et al., 2019; Bañón et al., 2020; Schwenk et al., 2021).”
Multimodal and Multilingual Embeddings for Large-Scale Speech Mining (2021, methodology): “This bitext mining has provided huge amounts of parallel data, in particular the ParaCrawl project (Esplà et al., 2019) or the CCMatrix initiative (Schwenk et al., 2021).”
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings (2020, methodology): “To demonstrate the effect with abundant parallel data we add up to 37M additional parallel sentences from ParaCrawl (Esplà et al., 2019) (see grey area).”
Ungoliant: An Optimized Pipeline for the Generation of a Very Large-Scale Multilingual Web Corpus (2021, unknown): “Of these, only OSCAR, Paracrawl, CCaligned and WikiMatrix are freely available and easily downloadable.”
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021, background): “Finally coarse-grained divergences might result from the process of automatically mining and aligning corpora from monolingual data (Fung and Cheung, 2004; Munteanu and Marcu, 2005), or web-scale parallel text (Smith et al., 2013; ElKishky et al., 2020; Esplà et al., 2019).”