Problem Formulation and Fairness

By Samir Passi, Solon Barocas. 2019

272 citations recorded by the source corpus.

Formulating data science problems is an uncertain and difficult process. It requires various forms of discretionary work to translate high-level objectives or strategic goals into tractable problems, necessitating, among other things, the identification of appropriate target variables and proxies. While these choices are rarely self-evident, normative assessments of data science projects often take them for granted, even though different translations can raise profoundly different ethical concerns. Whether we consider a data science project fair often has as much to do with the formulation of the problem as any property of the resulting model. Building on six months of ethnographic fieldwork with a corporate data science team---and channeling ideas from sociology and history of science, critical data studies, and early writing on knowledge discovery in databases---we describe the complex set of actors and activities involved in problem formulation. Our research demonstrates that the specification and operationalization of the problem are always negotiated and elastic, and rarely worked out with explicit normative considerations in mind. In so doing, we show that careful accounts of everyday data science work can help us better understand how and why data science problems are posed in certain ways---and why specific formulations prevail in practice, even in the face of what might seem like normatively preferable alternatives. We conclude by discussing the implications of our findings, arguing that effective normative interventions will require attending to the practical work of problem formulation.

Subfields: History and Philosophy of Science, Safety Research, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In History and Philosophy of Science

In Safety Research

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

Measurement and Fairness (2019, background): “ence|what Passi and Barocas call \problem formulation" in their ethnography of a data science company|captures organizational decisions, masking normative choices and downstream societal impacts [53]. The disconnect between the goals of an organization, constructs of interest, and their operati”
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, methodology): “defy deductive reasoning and definitive answers. Instead, engineering involves reconceptualizing a complex situation to facilitate analysis; with feedback loops impacting problem (re-)definition (see [109]), rather than focusing narrowly on problem solution. As goal-oriented models of reality, datas”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Several recent papers have demonstrated how social goals are, sometimes clumsily, abstracted and formulated to fit into a prediction task [23, 114, 48, 49, 94, 29, 113, 32, 100].”
Predictive Multiplicity in Classification (2019, background): “est error) from a collection of models that satisfy a certain constraints (e.g., linear classifiers with equal FPR between groups). Assuming that stakeholders agree on the formulation of this problem [38, 44] and that we can solve it, there is a key issue with this approach: there may be more than o”
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets (2020, methodology): “…and anthropology of sociotechnical systems has focused on the importance of analyzing interrogating unspoken work practices of technical experts (Passi & Barocas, 2019; Seaver, 2019), subject matter experts in acts of data repair (Sachs, 2019), and crowd laborers (Irani & Silberman, 2013;…”
Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics (2020, background): “…(ML) based interventions that have the potential of impacting the real lives of people; a step that involves determining the strategic goals driving the interventions and translating those strategic goals into tractable machine learning problems (Barocas et al., 2017; Passi & Barocas, 2019).”
Between Subjectivity and Imposition (2020, background): “Therefore, it is important that practitioners, i.e., corporations commissioning datasets and management at annotation companies, take steps to reflect, document, and communicate their subjective choices [38, 61, 65, 66, 72].”
Algorithmic realism: expanding the boundaries of algorithmic thought (2020, background): “considerations such as data availability and model accuracy, with “explicit normative considerations [rarely] in mind” [114].”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “Several recent papers have demonstrated how policy goals are, sometimes clumsily, abstracted and formulated to fit into a prediction task (Dobbe et al. 2018, Eckhouse et al. 2018, Green 2018, Green & Hu 2018, Ochigame et al. 2018, Silva & Kenney 2018, Passi & Barocas 2019, Selbst et al. 2019).”
Toward situated interventions for algorithmic equity: lessons from the field (2020, background): “Other scholars have pursued empirical case studies [19, 50, 52], field evaluations [10, 65], context-aware uses of datasets [46, 63], and questions of institutional access to real-world data [67].”
The Fallacy of AI Functionality (2022, background): “These engineering choices include deﬁning problem formulation [141], setting up evaluation criteria [118, 143], and determining a variety of other details [126, 142].”
Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “While theremay be further multiplicity-like problems stemming from the various ways that a nebulous real-world goal may be translated into predicting a specific target [44], we view these as out of scope for this paper.”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “The data that is collected can differ significantly from what occurs in the real world [77, 78, 119].”
Awareness in practice: tensions in access to sensitive attribute data for antidiscrimination (2019, background): “The fair ML research community has long reflected on the social and policy contexts of its work, recognizing legal tensions [8], historical parallels in prior debates over definitions of fairness [37], and the limitations of data-dependent problem formulation [64, 70].”
Mitigating bias in algorithmic hiring: evaluating claims and practices (2019, background): “As is the case in many domains, translating this to a feasible data-driven task forces our practitioner to make certain compromises [72].”