The ROOTS Search Tool: Data Transparency for LLMs

By Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurenccon, Gérard Dupont, Alexandra Sasha Luccioni, Yacine Jernite, Anna Rogers. 2023

42 citations recorded by the source corpus.

ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces: https://huggingface.co/spaces/bigscience-data/roots-search. We describe our implementation and the possible use cases of our tool.

Subfields: Artificial Intelligence, Management Science and Operations Research, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Evaluation & Benchmarks

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Different levels of access and description of the training data can help answer these questions with due consideration for privacy needs [183].”
StarCoder: may the source be with you! (2023, background): “This pressing need for understanding data (Mitchell et al., 2022) is being increasingly recognized and operationalized in the form of dataset inspection tools and toolkits (Akiki et al., 2023; Marone & Van Durme, 2023; Piktus et al., 2023).”
Data Portraits: Recording Foundation Model Training Data (2023, methodology): “See Table 2 for a size comparison with other tools, such as the ROOTS Search Tool BM25 index of Piktus et al. [30] on the ROOTS corpus [23]; and the Suffix Array of Lee et al. [24] on C4 [33].”
Multi-step Jailbreaking Privacy Attacks on ChatGPT (2023, background): “attention to the data quality and confidentiality of the web-sourced data (Piktus et al., 2023).”
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark (2023, methodology): “Without proper documentation and search tools like ROOTS (Piktus et al., 2023) it is very difficult for any researcher to actually know whether their datasets are compromised on a given model.”