DataLab: A Platform for Data Analysis and Intervention
By Yanghua Xiao, Jinlan Fu, Weizhe Yuan, Vijay Viswanathan, Zhoumianze Liu, Yixin Liu, Graham Neubig, Pengfei Liu. 2022
14 citations recorded by the source corpus.
Despite data’s crucial role in machine learning, most existing tools and research tend to focus on systems on top of existing data rather than how to interpret and manipulate data.In this paper, we propose DataLab, a unified data-oriented platform that not only allows users to interactively analyze the characteristics of data but also provides a standardized interface so that many data processing operations can be provided within a unified interface. Additionally, in view of the ongoing surge in the proliferation of datasets, DataLab has features for dataset recommendation and global vision analysis that help researchers form a better view of the data ecosystem. So far, DataLab covers 1,300 datasets and 3,583 of its transformed version, where 313 datasets support different types of analysis (e.g., with respect to gender bias) with the help of 119M samples annotated by 318 feature functions. DataLab is under active development and will be supported going forward. We have released a web platform, web API, Python SDK, and PyPI published package, which hopefully, can meet the diverse needs of researchers.
Subfields: Artificial Intelligence, Evaluation & Benchmarks, ML Systems & Hardware
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Douwe Kiela
- Mohit Bansal
- Pengfei Liu
- Nazneen Rajani
- Jinlan Fu
- Graham Neubig
- See-Kiong Ng
- Tristan Thrush
In Evaluation & Benchmarks
- See-Kiong Ng
- Jinlan Fu
- Pengfei Liu
- Tristan Thrush
- Douwe Kiela
- Angelina McMillan-Major
- Yacine Jernite
- Alexandra Sasha Luccioni
In ML Systems & Hardware
Citations in the corpus
- Measuring Data (2022, methodology): “Recent tools in this vein include CleanLab (2022), DataLab (Xiao et al., 2022) and Snorkel (Ratner et al., 2017), which provide solutions for analyzing and curating quality ML datasets, but are limited in scope to absent methods for systematic evaluation of the data.”
- Polyglot Prompt: Multilingual Multitask PrompTraining (2022, methodology): “…et al., 2021; Liu et al., 2020; Conneau et al., 2020; Conneau and Lample, 2019) enables different languages to be represented in a 1 We use DataLab (Xiao et al., 2022) to simplify the dataset loading process by performing the following pythonic command dataset = load_dataset(’poly_prompt’) unified…”