Institutionalizing ethics in AI through broader impact requirements

By Carina Prunkl, Carolyn Ashurst, Markus Anderljung, Helena Webb, Jan Leike, Allan Dafoe. 2021

75 citations recorded by the source corpus.

Turning principles into practice is one of the most pressing challenges of artificial intelligence (AI) governance. In this Perspective, we reflect on a governance initiative by one of the world’s largest AI conferences. In 2020, the Conference on Neural Information Processing Systems (NeurIPS) introduced a requirement for submitting authors to include a statement on the broader societal impacts of their research. Drawing insights from similar governance initiatives, including institutional review boards (IRBs) and impact requirements for funding applications, we investigate the risks, challenges and potential benefits of such an initiative. Among the challenges, we list a lack of recognized best practice and procedural transparency, researcher opportunity costs, institutional and social pressures, cognitive biases and the inherently difficult nature of the task. The potential benefits, on the other hand, include improved anticipation and identification of impacts, better communication with policy and governance experts, and a general strengthening of the norms around responsible research. To maximize the chance of success, we recommend measures to increase transparency, improve guidance, create incentives to engage earnestly with the process, and facilitate public deliberation on the requirement’s merits and future. Perhaps the most important contribution from this analysis are the insights we can gain regarding effective community-based governance and the role and responsibility of the AI research community more broadly. The Conference on Neural Information Processing Systems (NeurIPS) introduced a new requirement in 2020 that submitting authors must include a statement on the broader impacts of their research. Prunkl and colleagues discuss challenges and benefits of this requirement and propose suggestions to address the challenges.

Subfields: Cognitive Neuroscience, Health Informatics, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Verena Rieser
2. Gavin Abercrombie
3. Dirk Hovy
4. A. S. Bergman
5. Emily Dinan
6. Shannon L. Spruit
7. Y-Lan Boureau

In Health Informatics

1. Anders Søgaard
2. Patrick Hall

In Safety Research

1. Gavin Abercrombie
2. Dirk Hovy
3. Patrick Hall
4. Emily Dinan
5. Y-Lan Boureau

In AI Ethics, Fairness & Society

1. Anders Søgaard
2. Shannon L. Spruit
3. Gavin Abercrombie
4. Verena Rieser
5. Patrick Hall
6. Emily Dinan
7. Y-Lan Boureau
8. Zeerak Talat

Citations in the corpus

• Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “The process of envisioning impact – including both potential harms and benefits – is not straight-forward, as documented by Ovadya & Whittlestone (2019); Prunkl et al. (2021); Partnership on AI (2020); Partnership on AI (2021) among others, and it may not always be possible to estimate impact (§3.4).”
• Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbots (2023, background): “Moreover, ChatGPT has little information and has not (yet) been fully upgraded with information after 2021 [16,17].”
• Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “researchers to include statements about potential societal impacts [225] when submitting their work to journals or conferences.”
• Challenges and Strategies in Cross-Cultural NLP (2022, background): “Additionally, new works in AI ethics should also be considered (Nanayakkara et al., 2021; Hagendorff, 2020; Prunkl et al., 2021; Pitta et al., 1999; Mohamed et al., 2020).”