Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure

By Ben Hutchinson, Andrew Smart, A. Hanna, Emily L. Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, Margaret Mitchell. 2020

329 citations recorded by the source corpus.

Datasets that power machine learning are often used, shared, and reused with little visibility into the processes of deliberation that led to their creation. As artificial intelligence systems are increasingly used in high-stakes tasks, system development and deployment practices must be adapted to address the very real consequences of how model development data is constructed and used in practice. This includes greater transparency about data, and accountability for decisions made when developing it. In this paper, we introduce a rigorous framework for dataset development transparency that supports decision-making and accountability. The framework uses the cyclical, infrastructural and engineering nature of dataset development to draw on best practices from the software development lifecycle. Each stage of the data development lifecycle yields documents that facilitate improved communication and decision-making, as well as drawing attention to the value and necessity of careful data work. The proposed framework makes visible the often overlooked work and decisions that go into dataset creation, a critical step in closing the accountability gap in artificial intelligence and a critical/necessary resource aligned with recent work on auditing processes.

Subfields: Information Systems and Management, Management Information Systems, Management Science and Operations Research, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Management Information Systems

In Management Science and Operations Research

In Evaluation & Benchmarks

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Dataset creation is a value-laden process, yet those values are rarely made explicit (Hutchinson et al., 2021).”
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, background): “EN dataset. structured, task-speciﬁc NLP datasets, best practices have emerged around documenting the collection process, composition, intended uses, and other characteristics (Bender and Friedman, 2018; Gebru et al., 2018; Hutchinson et al., 2021).”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Researchers have advocated for increased transparency of datasets [26, 35], including developing tools to steer researcher intervention [8, 71].”
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation (2022, background): “Since large text-to-image models are foundation models [83] — enabling both a range of system applications as well as finetuning for specific image generation tasks — they act as a form of infrastructure which shapes our conceptions of what is both possible and desirable [84, 85].”
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, neutral): “Considerations • Consider designing and sharing a dataset maintenance plan [28].”
Datasets: A Community Library for Natural Language Processing (2021, background): “Datasets can be seen as a form of infrastructure (Hutchinson et al., 2021).”
Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development (2021, neutral): “While there are explicitly stated concerns about introducing human bias, in both the process of data collection and the process of annotation, there is little discussion about how bias, in terms of hermeneutic perspectives or as a matter of interpretation, is unavoidable in vision-based tasks.”
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021, background): “…freestanding datasheets documenting dataset releases of all kinds, with a focus on making potential harmful mismatches between data and application visible, and Hutchinson et al. (2021) argue along similar lines for a broader program of transparency and stakeholder engagement in data creation.”
Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI (2022, background): “Often, the goal has been to attain greater visibility into ML models and datasets by exposing source code [4], contribution trails [7], introducing ML-drive data analysis methods [18], and introducing diverse oversight [17].”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “It is common practice for classiﬁers of all kinds to be re-purposed far beyond their original contexts because building high quality datasets is challenging as well as under-valued [56].”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, methodology): “Although out of scope for this article, there is a need for more work which highlights the importance of data work for ML model development (Hutchinson et al., 2021; Sambasivan et al., 2021).”
Power to the People? Opportunities and Challenges for Participatory AI (2022, background): “The recent upswing in participatory tools have varied in approach and point in the machine learning product lifecycle, including: auditing methods for machine learning based interventions [48, 75], public consultation methods such as citizen juries [6, 83] or joint problem formulation [61], informat”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “Within the broader discussions around algorithmic fairness, our work is situated within a growing 75 body of scholarship that emphasizes the importance of understanding training data provenance and 76 context [19, 20] as a part of the model building process.”
Artsheets for Art Datasets (2021, background): “[23] Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell.”
Five sources of bias in natural language processing (2021, background): “However, as various papers have pointed out (Bender et al., 2021; Hutchinson et al., 2021), addressing data bias is not a ‘one-and-done’ exercise but requires continual monitoring throughout a data sets lifecycle.”