Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development

By Morgan Klaus Scheuerman, Emily L. Denton, A. Hanna. 2021

261 citations recorded by the source corpus.

Data is a crucial component of machine learning. The field is reliant on data to train, validate, and test models. With increased technical capabilities, machine learning research has boomed in both academic and industry settings, and one major focus has been on computer vision. Computer vision is a popular domain of machine learning increasingly pertinent to real-world applications, from facial recognition in policing to object detection for autonomous vehicles. Given computer vision's propensity to shape machine learning research and impact human life, we seek to understand disciplinary practices around dataset documentation - how data is collected, curated, annotated, and packaged into datasets for computer vision researchers and practitioners to use for model tuning and development. Specifically, we examine what dataset documentation communicates about the underlying values of vision data and the larger practices and goals of computer vision as a field. To conduct this study, we collected a corpus of about 500 computer vision datasets, from which we sampled 114 dataset publications across different vision tasks. Through both a structured and thematic content analysis, we document a number of values around accepted data practices, what makes desirable data, and the treatment of humans in the dataset construction process. We discuss how computer vision datasets authors value efficiency at the expense of care; universality at the expense of contextuality; impartiality at the expense of positionality; and model work at the expense of data work. Many of the silenced values we identify sit in opposition with social computing practices. We conclude with suggestions on how to better incorporate silenced values into the dataset creation and curation process.

Subfields: Computer Vision and Pattern Recognition, Safety Research, Evaluation & Benchmarks, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety Research

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Unfortunately, none of these suggestions are typically followed: Scheuerman et al. (2021) analyzed 114 computer vision datasets and find that their creation process values efficiency at the expense of care and that they typically aim to be as universal as possible without nuanced understanding of…”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “There is no universal evaluation by which to evaluate a model’s performance, and any evaluation metrics should be used with deployment context in mind [210, 191].”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, background): “For example, public and academic discourse regarding appropriate use of public data has raised concerns regarding data subject awareness and consent [24, 18, 63, 45].”
Measuring Data (2022, background): “…as the scale of datasets used in machine learning has drastically increased in recent years, measuring practices have fallen behind as values of efﬁciency and approaches that prioritize quantity over considerations of other dataset qualities have become more common place (Scheuerman et al., 2021).”
BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model (2022, background): “…(Sambasivan et al., 2021), and to how prioritizing efﬁciency and technical performance comes at the expense of social considerations for datasets (Scheuerman et al., 2021; Birhane et al., 2021b); including over-relying on automatic curation that fails to examine the additional biases it…”
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, background): “For example, relatively little attention is given or documented about an-notator positionality—how annotator social identity shapes their understanding of the world [18, 48].”
Multimodal datasets: misogyny, pornography, and malignant stereotypes (2021, background): “…of large scale computer vision datasets contains various broad problems including curation biases, inclusion of problematic content in the images, the questionable approaches of associating these images with offensive and non-imageable labels, as well as the gradual erosion of privacy [4, 5].”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “ance on crowdworkers, was seen as a boon to machine learning.(1,3,6,7) The enormous scale of such datasets has been mythologized as beneficial to the perceived generality of trained systems,(7) but they continue to be impacted by the limitations and biases that impact all datasets.”
Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022, background): “Very recent research — most published within the last year — has begun to analyze dataset values [18, 40], question assumptions around dataset use [100], unpack what is represented in ML datasets [43, 88, 124] and establish basics of how an organized dataset lifecycle might proceed [74].”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, background): “A suitable mechanism for assigning and updating identiﬁers could be based on the DOI (digital object identiﬁer) system, which has existed for decades but has been largely overlooked by the ML community [90].”
Artsheets for Art Datasets (2021, background): “Unstandardized dataset documentation,and maintenance frameworks have been identiﬁed as a key barrier to reproducibility and accountability [21, 22, 23, 15].”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, background): “Although we have limited the scope of our work to the contents of visual datasets itself, there are much broader questions of fairness and justice to be considered regarding the role that datasets play (Denton et al., 2020; Paullada et al., 2020; Scheuerman et al., 2021; Peng et al., 2021).”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “Such datasets have various potentially problematic issues such as consent and subject awareness (Paullada et al., 2021; Dulhanty, 2020; Scheuerman et al., 2021).”
Evaluation Gaps in Machine Learning Practice (2022, methodology): “To reduce the gap introduced by the Consequentialism Assumption, evaluate the processes that led to the creation of the model, including how datasets were constructed [150].”
Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research (2021, background): “Despite widespread recognition that datasets are critical to the advancement of the field, careful 122 dataset development is often undervalued and disincentivized, especially relative to algorithmic 123 contributions [32, 12].”
Leakage and the reproducibility crisis in machine-learning-based science (2023, background): “Access to good-quality data is essential for creating ML models.(67,68) Issues with the quality of the dataset could affect the results of ML-based science.”
Advances, challenges and opportunities in creating data for trustworthy AI (2022, background): “It is important for data documentation to capture the lifecycle and sociotechnical context of the data where applicable 45,46 .”