Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?

By Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé Iii, Miroslav Dud'ik, Hanna M. Wallach. 2018

1,021 citations recorded by the source corpus.

The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention. A surge of recent work has focused on the development of algorithmic tools to assess and mitigate such unfairness. If these tools are to have a positive impact on industry practice, however, it is crucial that their design be informed by an understanding of real-world needs. Through 35 semi-structured interviews and an anonymous survey of 267 ML practitioners, we conduct the first systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems. We identify areas of alignment and disconnect between the challenges faced by teams in practice and the solutions proposed in the fair ML research literature. Based on these findings, we highlight directions for future ML and HCI research that will better address practitioners' needs.

Subfields: Computer Science Applications, Safety Research, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

Re-imagining Algorithmic Fairness in India and Beyond (2021, background): “Biases in models are frequently attributed to biased datasets, presupposing the possibility of achieving fairness by “fixing” the data [89].”
Measurement and Fairness (2019, background): “s [53]. The disconnect between the goals of an organization, constructs of interest, and their operationalization become apparent when data scientists diagnose known sources of fairness-related harms [32]. Furthermore, a focus on \fairness" in systems that lack validity and reliability generall”
Datasheets for datasets (2018, background): “[13] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, and Hanna M.”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “As one data scientist put it, “if it is available to us, we ingest it” [Holstein et al., 2019].”
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, methodology): “ for cultural shifts (Section 5). 2 DATASETS AS INFRASTRUCTURE ML fairness practitioners have identified collection and curation of representative data as the most common way to mitigate model biases [71], echoing the broader tendency within ML communities to valorize larger datasets as necessarily ”
FairSight: Visual Analytics for Fairness in Decision Making (2019, background): “The challenge is how to facilitate such an understanding as many practitioners do not fully recognize how every step in the process could potentially lead to biased decision making [21].”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “…biases are at times overlooked due to the personal blind spots of those working with social data (Holstein et al., 2019; West et al., 2019 how to properly collect or curate high-quality datasets, though critical in most application scenarios, also remains a lingering issue (Holstein et al., 2019).”
Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, methodology): “We emphasize that the effective usage of our methods requires careful thought about the broader social context surrounding the setting of interest (Selbst et al., 2019; Holstein et al., 2019).”
Explainable machine learning in deployment (2019, background): “[28], we study how industry practitioners look at and deploy explainable ML.”
What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes (2019, background): “And, in other domains, protected attributes are often not available (Holstein et al., 2019).”
Disembodied Machine Learning: On the Illusion of Objectivity in NLP (2021, background): “This can help explain why ML practitioners frequently call for ‘more’ and ‘more diverse’ data (Holstein et al., 2019) to”
Algorithmic Fairness (2020, background): “These lines of evidence and concerns about algorithmic fairness have led to growing interest in the literature on defining, evaluating and improving fairness in AI algorithms (see, for example, [15, 37, 57, 68]).”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “There is a growing literature of work studying fairness practitioners themselves that has focused on the needs of public sector practitioners [82], the needs of private sector practitioners [41] and the organizational patterns they fit within [67].”