Not Just Bigger : Towards Better-Quality Web Corpora

By Yannick Versley. 2012

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Iryna Gurevych
2. Daphne Ippolito
3. Nicholas Carlini
4. Katherine Lee
5. Chris Callison-Burch
6. Ivan Habernal
7. Chiyuan Zhang
8. Mohit Bansal

In Evaluation & Benchmarks

1. Ivan Habernal
2. Chiyuan Zhang
3. Iryna Gurevych
4. Chris Callison-Burch
5. Andrew Nystrom
6. Douglas Eck
7. Nicholas Carlini
8. Katherine Lee

Citations in the corpus

• Deduplicating Training Data Makes Language Models Better (2021, methodology): “MinHash (Broder, 1997) is an approximate matching algorithm widely used in large-scale deduplication tasks (Versley and Panchenko, 2012; Gabriel et al., 2018; Gyawali et al., 2020), including to deduplicate the training set for a large Chinese-language LM (Zeng et al., 2021).”
• C4Corpus: Multilingual Web-size Corpus with Free License (2016, background): “Versley and Panchenko (2012) crawled the Web with the focus on the German sites in two categories: news-style content and general Web content.”