Addressing "Documentation Debt" in Machine Learning: A Retrospective Datasheet for BookCorpus

By Jack Bandy, Nicholas Vincent. 2021

39 citations recorded by the source corpus.

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Noah A. Smith
2. Hannaneh Hajishirzi
3. Eric Wallace
4. Luke S. Zettlemoyer
5. Percy Liang
6. Sewon Min
7. Yejin Choi
8. Dan Jurafsky

Citations in the corpus

• Foundation Models and Fair Use (2023, methodology): “For example, Bandy & Vincent (2021) pointed out that the BookCorpus contains copyrighted data under restrictive licenses and has been used to train large foundation models including GPT-3 (Brown et al., 2020) and BERT (Devlin et al., 2018).”
• Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection (2022, background): “, 2015), which consists of 7,185 self-published novels, a dataset criticized for copyright violation, imbalanced representation, and lack of documentation (Bandy and Vincent, 2021).”
• SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore (2023, background): “For example, sources such as GitHub repositories and book collections typically contain text with highly restrictive licenses (Bandy & Vincent, 2021).”