Overcoming Failures of Imagination in AI Infused System Development and Deployment

By M. Boyarskaya, Alexandra Olteanu, Kate Crawford. 2020

63 citations recorded by the source corpus.

NeurIPS 2020 requested that research paper submissions include impact statements on "potential nefarious uses and the consequences of failure." However, as researchers, practitioners and system designers, a key challenge to anticipating risks is overcoming what Clarke (1962) called 'failures of imagination.' The growing research on bias, fairness, and transparency in computational systems aims to illuminate and mitigate harms, and could thus help inform reflections on possible negative impacts of particular pieces of technical work. The prevalent notion of computational harms -- narrowly construed as either allocational or representational harms -- does not fully capture the open, context dependent, and unobservable nature of harms across the wide range of AI infused systems.The current literature focuses on a small range of examples of harms to motivate algorithmic fixes, overlooking the wider scope of probable harms and the way these harms might affect different stakeholders. The system affordances may also exacerbate harms in unpredictable ways, as they determine stakeholders' control(including of non-users) over how they use and interact with a system output. To effectively assist in anticipating harmful uses, we argue that frameworks of harms must be context-aware and consider a wider range of potential stakeholders, system affordances, as well as viable proxies for assessing harms in the widest sense.

Subfields: Artificial Intelligence, Safety Research, AI Ethics, Fairness & Society, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hal Daumé Iii
2. Su Lin Blodgett
3. Adam Trischler
4. Negar Rostamzadeh
5. Hanna M. Wallach
6. Alexandra Olteanu
7. Robert Sim
8. Gilsinia Lopez

In Safety Research

1. Su Lin Blodgett
2. Alexandra Olteanu
3. Negar Rostamzadeh
4. Andrew Smart
5. Hal Daumé Iii
6. Hanna M. Wallach

In AI Ethics, Fairness & Society

1. Hal Daumé Iii
2. Kathryn Henne
3. Andrew Smart
4. Hanna M. Wallach
5. P. Nicholas
6. Gurleen Virk
7. Jess Gallegos
8. Emilio Garcia

In Human-Computer Interaction

1. Andrew Smart
2. Hal Daumé Iii
3. Hanna M. Wallach
4. Su Lin Blodgett

Citations in the corpus

• Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets (2021, background): “…modeling framework can provide analytical clarity and a scaffold for future work in this direction. be inherently beneficial with little to no potential for harm, and thus often (paradoxically) fails to examine its limitations or possible unintended negative consequences (Boyarskaya et al., 2020).”
• Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “As such, the epistemological choices made in developing harm taxonomies focus practitioner attention on particular areas over others [34].”
• Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications (2022, background): “, 2021) or even to imagine adverse impacts (Boyarskaya et al., 2020).”
• Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “In alignment with prior calls to anticipate computational and contextual harms [40, 151], we synthesize insights on harms from”