Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI

By Michael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, Hanna M. Wallach. International Conference on Human Factors in Computing Systems, 2020

520 citations recorded by the source corpus.

Many organizations have published principles intended to guide the ethical development and deployment of AI systems; however, their abstract nature makes them difficult to operationalize. Some organizations have therefore produced AI ethics checklists, as well as checklists for more specific concepts, such as fairness, as applied to AI systems. But unless checklists are grounded in practitioners' needs, they may be misused. To understand the role of checklists in AI ethics, we conducted an iterative co-design process with 48 practitioners, focusing on fairness. We co-designed an AI fairness checklist and identified desiderata and concerns for AI fairness checklists in general. We found that AI fairness checklists could provide organizational infrastructure for formalizing ad-hoc processes and empowering individual advocates. We highlight aspects of organizational culture that may impact the efficacy of AI fairness checklists, and suggest future design directions.

Subfields: Cognitive Neuroscience, Information Systems and Management, Safety Research

View Paper: https://www.semanticscholar.org/paper/58bb221c1e375f254826b7b7341f74057e87676c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Information Systems and Management

In Safety Research

Citations in the corpus

The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence (2022, background): “Additionally, there is no global agreement on standards when it comes to dataset quality investigation, and this is exacerbated by the ‘move fast’ culture endemic in the tech industry that often prioritizes rapid deployment above all else [19].”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, methodology): “…data stewardship practices in all steps of the dataset life cycle, ranging from proper documentation at the data creation stage via checklists [58] and data sheets [34] to the study of dataset usage and genealogy [32, 50], licensing and auditing [11, 18, 78, 88], as well as data stewardship…”
Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes (2022, background): “Studies to help AI practitioners improve fairness [22, 34], such as checklists to solve organizational challenges [49], are often based on the needs of AI practitioners in private companies, but some studies also focus on the needs of public sector [80] or academic institutions [61].”
Intersectional Bias in Causal Language Models (2021, background): “In response to these concerns, scholars as well as companies such as Microsoft(2) and Google(3) have developed fairness principles and practices for AI development and deployment [43].”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “Researchers have also developed “ethics methods” [150] for practitioners to identify and mitigate sociotechnical harms, including statistical assessment [135, 145], software”
Algorithm fairness in artificial intelligence for medicine and healthcare (2023, background): “Such binary partitioning of the roles may lead to poor practices, as fairness checklists are often too broad or abstractive, and are not co-designed with engineers to address problem-specific and technical challenges for the achievement of fairness 256 .”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “There is momentum among practitioners to develop practices to better identify and minimize harms from algorithmic systems (e.g., [24, 53, 116, 125].”
Between Subjectivity and Imposition (2020, background): “Recent research work has highlighted the importance of structured disclosure documents that should accompany datasets [7, 38, 39, 47, 55, 78].”
Critical Perspectives on Governance Mechanisms for AI/ML Systems (2020, background): “…this and much related scholarship is the necessity of foregrounding and centering the expertise of communities affected by AI/ML systems in their design and use (Costanza-Chock, 2018; Forlano & Mathew, 2013; Madaio, Stark, Wortman Vaughan, & Wallach, 2020; Sloan, Moss, Awomolo, & Forlano, 2020).”
Six Human-Centered Artificial Intelligence Grand Challenges (2023, unknown): “When it comes to fairness and ethics, it is helpful that there are already several relevant principles and checklists available, however, it is very difficult for practitioners to successfully apply them in the numerous decisions they have to make on a daily basis (Madaio et al., 2020).”
Design Principles for Generative AI Applications (2024, background): “…guidelines focus on specifc types of AI technologies (e.g. text-to-image models [104]), specifc domains of use (e.g. creative writing [24]), or specifc issues regarding the use of AI, including ethics [11, 59, 69, 72], fairness [110], human rights [50], explainabil-ity [119], and user trust [186].”
Trustworthy AI: From Principles to Practices (2021, methodology): “Despite the simplicity of the abstract criteria outlined in the previous section, fairness can manifest in many different forms following these principles (see Reference [66, 356] for comprehensive surveys, and Reference [228] for AI ethics’ checklists).”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “haracteristics of trained AI models. The examples of such documentations are: data statements [6], datasheets for datasets [18], model cards [54], reproducibility checklists [56], fairness checklists [48], and factsheets [2]. We illustrate the connection between these documentations and the European”
Overcoming Failures of Imagination in AI Infused System Development and Deployment (2020, background): “This phenomenon occurs even when the process of identifying risks is guided by extensive, well researched check-lists (Madaio et al., 2020; Kiran et al., 2015; Wong et al., 2020)—as they tend to be general and fail to account for differences between technologies, applications, and stakeholders…”