Dataset Debt in Biomedical Language Modeling
By Wojciech Kusa. 2022
Large-scale language modeling and natural language prompting have demonstrated exciting capabilities for few and zero shot learning in NLP. However, translating these successes to specialized domains such as biomedicine remains challenging, due in part to biomedical NLP’s significant dataset debt – the technical costs associated with data that are not consistently documented or easily incorporated into popular machine learning frameworks at scale. To assess this debt, we crowdsourced curation of datasheets for 167 biomedical datasets. We find that only 13% of datasets are available via programmatic access and 30% lack any documentation on licensing and permitted reuse. Our dataset catalog is available at: https://tinyurl.com/bigbio22.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Mohit Bansal
- Yonatan Belinkov
- Yejin Choi
- Noah A. Smith
- Sebastian Gehrmann
- Graham Neubig
- Samuel R. Bowman
- Percy Liang
Citations in the corpus
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “In addition, a few groups focused on the evaluation of LLMs and dataset development in specific domains, such as biomedical texts (Fries et al., 2022b) and historical texts (De Toni et al., 2022).”