Dirt Cheap Web-Scale Parallel Text from the Common Crawl

By Jason R. Smith, Herve Saint-Amand, Magdalena Plamada, Philipp Koehn, Chris Callison-Burch, Adam Lopez. 2013

164 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, background): “, 2014], for commonsense reasoning [Trinh and Le, 2018], for mining parallel texts for machine translation [Smith et al., 2013], as a pre-training dataset [Baevski et al.”
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation (2020, background): “…with the English pronouns. glish was most recently contested at WMT’13 (Bojar et al., 2013), which oﬀered participants 14,980,513 sentence pairs from Eu-roparl 3 (Habash et al., 2017), Common Crawl (Smith et al., 2013), the United Nations corpus 4 (Ziemski et al., 2016), and news commentary.”
Analyzing Uncertainty in Neural Machine Translation (2018, background): “To save time and effort, it is common to augment high quality human translated corpora with lower quality web crawled data (Smith et al., 2013).”
The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English (2019, background): “…in recent years thanks to improvements in modeling, and in particular neural models (Sutskever et al., 2014; Bahdanau et al., 2015; Gehring et al., 2016; Vaswani et al., 2017), as well as the availability of large parallel corpora for training (Tiedemann, 2012; Smith et 2013; Bojar et al., 2017).”
Exploring neural transducers for end-to-end speech recognition (2017, methodology): “ng the other models is the same 4-gram LM available for this benchmark from the Kaldi receipe [20]. The language model used by all models in Table 3 is built from a sample of the common crawl dataset [26]. Model speciﬁcation. All models in Tables 1 and 3 are tuned independent of each other - we perf”
A Massive Collection of Cross-Lingual Web-Document Pairs (2019, background): “Other works (Smith et al., 2013) have mined Common Crawl for bitexts for machine translation.”
Two New Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English (2019, background): “…F-75005, Paris, France. provements in modeling, and in particular neural models (Sutskever et al., 2014; Bahdanau et al., 2015; Gehring et al., 2016; Vaswani et al., 2017), as well as the availability of large parallel corpora for training (Tiedemann, 2012; Smith et al., 2013; Bojar et al., 2017).”
C4Corpus: Multilingual Web-size Corpus with Free License (2016, background): “Relevant research that exploits CommonCrawl includes mining parallel texts for machine translation by Smith et al. (2013) or extracting n-grams and building language models by Buck et al. (2014).”
Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle* (2021, background): “times as many HE than SHE in Europarl (Koehn 2005) and the Common Crawl (Smith et al. 2013), six times as many in News Commentaries (Tiedemann 2012), and fifty times”
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021, background): “Finally coarse-grained divergences might result from the process of automatically mining and aligning corpora from monolingual data (Fung and Cheung, 2004; Munteanu and Marcu, 2005), or web-scale parallel text (Smith et al., 2013; El-Kishky et al., 2020; Espl ` a et al., 2019).”
The AFRL-MITLL WMT15 System: There’s More than One Way to Decode It! (2015, methodology): “The Commoncrawl corpus was further processed as in Schwartz et al. (2014) to exclude wrong-language text and to normalize mixed-alphabet spellings.”
Multimodal Learning for Web Information Extraction (2017, methodology): “We derive our text and image corpus based on the Common Crawl dataset [25] that is publicly available on Amazon S3.”
DFKI’s experimental hybrid MT system for WMT 2015 (2015, unknown): “9 (Bojar et al., 2014), Commoncrawl (Smith et al., 2013) and MultiUN (Eisele and Chen, 2010).”
A Corpus and Semantic Parser for Multilingual Natural Language Querying of OpenStreetMap (2016, methodology): “Here we employ its standard features plus additional sparse features and the C OMMON C RAWL 6 (Smith et al., 2013) corpus to built the baseline SMT system.”
The AFRL-MITLL WMT16 News-Translation Task Systems (2016, methodology): “These six million lines represent 0.17% of the 3.6 billion lines of data in the English portion of the Common Crawl.”
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation (2020, background): “the availability of large parallel corpora for training such systems (Smith et al., 2013; Kocmi and Bojar, 2017; Tiedemann, 2012).”