Critique and Contribute: A Practice-Based Framework for Improving Critical Data Studies and Data Science

By Gina Neff, Anissa Tanweer, Brittany Fiore-Gartland, Laura Osburn. 2017

148 citations recorded by the source corpus.

Abstract What would data science look like if its key critics were engaged to help improve it, and how might critiques of data science improve with an approach that considers the day-to-day practices of data science? This article argues for scholars to bridge the conversations that seek to critique data science and those that seek to advance data science practice to identify and create the social and organizational arrangements necessary for a more ethical data science. We summarize four critiques that are commonly made in critical data studies: data are inherently interpretive, data are inextricable from context, data are mediated through the sociomaterial arrangements that produce them, and data serve as a medium for the negotiation and communication of values. We present qualitative research with academic data scientists, “data for good” projects, and specialized cross-disciplinary engineering teams to show evidence of these critiques in the day-to-day experience of data scientists as they acknowledge and grapple with the complexities of their work. Using ethnographic vignettes from two large multiresearcher field sites, we develop a set of concepts for analyzing and advancing the practice of data science and improving critical data studies, including (1) communication is central to the data science endeavor; (2) making sense of data is a collective process; (3) data are starting, not end points, and (4) data are sets of stories. We conclude with two calls to action for researchers and practitioners in data science and critical data studies alike. First, creating opportunities for bringing social scientific and humanistic expertise into data science practice simultaneously will advance both data science and critical data studies. Second, practitioners should leverage the insights from critical data studies to build new kinds of organizational arrangements, which we argue will help advance a more ethical data science. Engaging the insights of critical data studies will improve data science. Careful attention to the practices of data science will improve scholarly critiques. Genuine collaborative conversations between these different communities will help push for more ethical, and better, ways of knowing in increasingly datum-saturated societies.

Subfields: Information Systems, Management Information Systems, Management Science and Operations Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Management Information Systems

In Management Science and Operations Research

In AI Ethics, Fairness & Society

Citations in the corpus

Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward (2020, background): “Even the data on which an algorithm is trained on are not an objective truth, they are dependent upon the context in which they have been produced (Neff et al., 2017).”
Problem Formulation and Fairness (2019, background): “[33], through ethnographic work on academic data science research, show how data scientists often “acknowledge their interpretive contributions” and “use data to surface and negotiate social values.”
Data Science as Political Action: Grounding Data Science in a Politics of Justice (2018, background): “In practice, computer scientists are “diverse and ambivalent characters”[25] who engage in “nuanced, contextualized, and reflexive practices”[26].”
Human-AI Collaboration in Data Science (2019, background): “HCI and CSCW researchers have studied how data scientists work with their data (e.g., [11, 46, 48, 50, 53, 54]).”
Algorithmic realism: expanding the boundaries of algorithmic thought (2020, background): “practice, computer scientists are “diverse and ambivalent characters” [133] who blend formalist and realist methods, engaging in “nuanced, contextualized, and re exive practices” [104] as they “continuously straddle the competing demands of formal abstraction and empirical contingency” [115].”
"The human body is a black box": supporting clinical decision-making with deep learning (2019, background): “That is, how, why, and in what context data science problems are formulated as problems to be solved is neither value-neutral nor self-obvious [39].”
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, background): “ as knowledge of statistics. This aligns with the prescriptions of Neffet al., who ask data scientists to consider several interpretative and value-laden critiques of their data while working with it [105]. 3.3 Datasets are models of reality Central to engineering is the concept of engineering model”