Fairness and Abstraction in Sociotechnical Systems

By Andrew D. Selbst, D. boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, Janet Vertesi. 2019

1,478 citations recorded by the source corpus.

A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science---such as abstraction and modular design---are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones.

Subfields: Computer Science Applications, Safety Research, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, background): “Indeed, the performance of an AI system in isolation may not reveal much about the ultimate performance of a sociotechnical system that involves, for example, human discretion and judgment (Selbst et al. 2019).”
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “ment of NLP systems should also include the socio-technical considerations for various stakeholders impacted by the deployment, both directly and indirectly, as well as voluntarily and involuntarily (Selbst et al., 2019;Bender,2019), accounting for long-term impacts (Liu et al.,2019;D’Amour et al.,2”
Measurement and Fairness (2019, background): “modeling brings the biggest value to fairness. In practice, evaluation of computational systems can fail to characterize potential harms which are otherwise captured by failures of construct validity [27, 53, 59]. The language of measurement, with the tools of construct validity and reliability, pro”
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, background): “This contributes to the portability trap [51]: a “failure to understand how repurposing algorithmic solutions designed for one social context”
Lessons from archives: strategies for collecting sociocultural data in machine learning (2019, background): “more issues related to ML systems, many have started calling for an interdisciplinary approach to understanding and tackling these issues [58].”
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, background): “infrastructure. It is deep in the training of computer programmers to take infrastructure for granted; indeed, the use of abstractions in software libraries encourages this (the “portability trap” of [128]). Although software developers and model developers both need to “get things done” by working ”
Fair Regression: Quantitative Definitions and Reduction-based Algorithms (2019, methodology): “As with other algorithmic fairness tools, in order to successfully use our algorithms in practice, it is essential to consider the societal context of the application (Selbst et al., 2018).”
Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, methodology): “We emphasize that the effective usage of our methods requires careful thought about the broader social context surrounding the setting of interest [19, 20].”
PaLM 2 Technical Report (2023, methodology): “These measures are a limited approximation, and are by no means exhaustive or holistically complete (Raji et al., 2021; Selbst et al., 2019).”
Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing (2020, background): “Traditional metrics for artificial intelligence like loss may conceal fairness concerns, social impact risks or abstraction errors [69].”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, background): “Yet, technical approaches can meaningfully contribute when they appropriately reflect the values and needs of relevant stakeholders and impacted groups (Selbst et al. 2019).”
Towards a critical race methodology in algorithmic fairness (2019, neutral): “rent algorithmic fairness frameworks [53, 108].”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, background): “Solutions oriented around balancing representation of sociodemographic groups within ML datasets often re ﬂ ect a focus on “ fairness ” at the wrong level of abstraction (Selbst et al., 2019).”