Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents

By Meg Young, Lassana Magassa, Batya Friedman. 2019

79 citations recorded by the source corpus.

To be successful, policy must anticipate a broad range of constituents. Yet, all too often, technology policy is written with primarily mainstream populations in mind. In this article, drawing on Value Sensitive Design and discount evaluation methods, we introduce a new method—Diverse Voices—for strengthening pre-publication technology policy documents from the perspective of underrepresented groups. Cost effective and high impact, the Diverse Voices method intervenes by soliciting input from “experiential” expert panels (i.e., members of a particular stakeholder group and/or those serving that group). We first describe the method. Then we report on two case studies demonstrating its use: one with a white paper on augmented reality technology with expert panels on people with disabilities, people who were formerly or currently incarcerated, and women; and the other with a strategy document on automated driving vehicle technologies with expert panels on youth, non-car drivers, and extremely low-income people. In both case studies, panels identified significant shortcomings in the pre-publication documents which, if addressed, would mitigate some of the disparate impact of the proposed policy recommendations on these particular stakeholder groups. Our discussion includes reflection on the method, evidence for its success, its limitations, and future directions.

Subfields: Global and Planetary Change, Management of Technology and Innovation, Sociology and Political Science, AI Ethics, Fairness & Society, Human-Computer Interaction, Social Sciences & Humanities

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Global and Planetary Change

In Management of Technology and Innovation

In Sociology and Political Science

In AI Ethics, Fairness & Society

In Human-Computer Interaction

In Social Sciences & Humanities

Citations in the corpus

Toward Gender-Inclusive Coreference Resolution (2019, background): “In comparison to [Zhao et al. 2018] and [Rudinger et al.”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, methodology): “Design There is now also a growing understanding of approaches for meaningful community-engaged research (Mikesell et al. 2013), using frameworks like the IEEE Ethically Aligned Design (IEEE Global Initiative 2016), technology policy design frameworks like Diverse Voices (Young et al. 2019) and mech”
Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context (2020, result): “variants such as community based system dynamics (CBSD) are analogous to other efforts in the ML fairness community [7, 111] seeking to aid developers and researchers who desire working as”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021, background): “techniques as envisioning cards [48], the development of value scenarios [90], and working with panels of experiential experts [152].”
Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle* (2021, background): “It may be the case that to truly build gender inclusive datasets and systems, we need to hire or consult experiential experts (Patton et al. 2019; Young, Magassa, and Friedman 2019).”
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice (2023, background): “Yet, it is worth grappling seriously with the implications that the use of proxies has as a “labor-saving device” [154] or as a form of ‘discount’ participation (following Borning et al. [34], Young et al. [225]).”
ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles (2019, methodology): “…that diverse perspectives — especially those from communities historically excluded from technology decision-making — contribute to any ABOUT ML recommendations, we are engaging with the Tech Policy Lab at the University of Washington to conduct Diverse Voices panels for the ABOUT ML project [18].”
Toward situated interventions for algorithmic equity: lessons from the field (2020, methodology): “As another part of our evaluative strategy and co-design process, we presented the Toolkit to three panels using the Diverse Voices method [66].”