Where are human subjects in Big Data research? The emerging ethics divide

By Jacob Metcalf, Kate Crawford. 2016

357 citations recorded by the source corpus.

There are growing discontinuities between the research practices of data science and established tools of research ethics regulation. Some of the core commitments of existing research ethics regulations, such as the distinction between research and practice, cannot be cleanly exported from biomedical research to data science research. Such discontinuities have led some data science practitioners and researchers to move toward rejecting ethics regulations outright. These shifts occur at the same time as a proposal for major revisions to the Common Rule—the primary regulation governing human-subjects research in the USA—is under consideration for the first time in decades. We contextualize these revisions in long-running complaints about regulation of social science research and argue data science should be understood as continuous with social sciences in this regard. The proposed regulations are more flexible and scalable to the methods of non-biomedical research, yet problematically largely exclude data science methods from human-subjects regulation, particularly uses of public datasets. The ethical frameworks for Big Data research are highly contested and in flux, and the potential harms of data science research are unpredictable. We examine several contentious cases of research harms in data science, including the 2014 Facebook emotional contagion study and the 2016 use of geographical data techniques to identify the pseudonymous artist Banksy. To address disputes about application of human-subjects research ethics in data science, critical data studies should offer a historically nuanced theory of “data subjectivity” responsive to the epistemic methods, harms and benefits of data science and commerce.

Subfields: Artificial Intelligence, Public Health, Environmental and Occupational Health, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Public Health, Environmental and Occupational Health

In Safety Research

Citations in the corpus

Critical data studies: An introduction (2016, background): “There is evidence that data are surreptitiously extracted from data subjects (Hauge et al., 2016; Metcalf and Crawford, 2016), hijacked to serve agendas that benefit research and industry (Ioannidis, 2005, 2016), and compromised by the interests of not only powerful business organizations but also h”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “Research on human subjects is regulated by law in many jurisdictions; and given that data elements in social datasets represent people or groups of people (Varshney, 2015; Diaz, 2016), research on social data is, arguably, human subjects research (Metcalf and Crawford, 2016).”
Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards (2021, methodology): “Reproducibility The reproducibility section of the card combines elements of Mitchell et al. (2019)’s model cards and Dodge et al.”
People’s Councils for Ethical Machine Learning (2018, background): “The emergence of big data has fueled the debate about the centrality of human subjects to ethical assessments (Metcalf & Crawford, 2016).”
Research Dilemmas with Behavioral Big Data (2017, neutral): “, information privacy and data discrimination) and are therefore not defined as risks by Common Rule.(7) Consent is often obtained through a long and largely unread online ‘‘data use policy’’ page, which gives a blanket coverage for all future company uses of the data; such policies are often modifi”
Ten simple rules for responsible big data research (2017, background): “’s [8] use of property records and geographic profiling techniques to allegedly identify the pseudonymous artist Banksy [9].”
Fairness in Machine Learning: A Survey (2020, neutral): “[118, 269, 49, 72, 15, 33, 96, 217, 297, 41, 42, 203, 181]), governments (e.”
Exploring or Exploiting? Social and Ethical Implications of Autonomous Experimentation in AI (2016, neutral): “ently “low risk”—a contentious issue in and of itself [14]—it”
Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches (2021, background): “…is difﬁcult and inaccessible for the many groups without access to real users interacting with a real system, and privacy concerns can also limit the ethical collection and distribution of such benchmarks (Richards and King, 2014; Metcalf and Crawford, 2016; Paullada et al., 2020, inter alia ).”