Truth from the machine: artificial intelligence and the materialization of identity

By Os Keyes, Zoë Hitzig, M. Blell. 2021

Critics now articulate their worries about the technologies, social practices and mythologies that comprise Artificial Intelligence (AI) in many domains. In this paper, we investigate the intersection of two domains of criticism: identity and scientific knowledge. On one hand, critics of AI in public policy emphasise its potential to discriminate on the basis of identity. On the other hand, critics of AI in scientific realms worry about how it may reorient or disorient research practices and the progression of scientific inquiry. We link the two sets of concerns—around identity and around knowledge—through a series of case studies. In our case studies, about autism and homosexuality, AI figures as part of scientific attempts to find, and fix, forms of identity. Our case studies are instructive: they show that when AI is deployed in scientific research about identity and personality, it can naturalise and reinforce biases. The identity-based and epistemic concerns about AI are not distinct. When AI is seen as a source of truth and scientific knowledge, it may lend public legitimacy to harmful ideas about identity.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

1. Yejin Choi
2. Ximing Lu
3. Iason Gabriel
4. Ali Farhadi
5. Rowan Zellers
6. Po-Sen Huang
7. Youngjae Yu
8. Amelia Glaese

Citations in the corpus

• Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, neutral): “Misgendering persons and the associated harms have been studied in contexts of computer vision (Keyes, 2018) and human-computer interaction (Keyes et al., 2021), which highlight its adverse impact on the mental health of non-binary individuals.”
• MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “The ‘knowledge’ that our model learns should be viewed as situated within YouTube [66], which has numerous biases (that we will discuss next).”
• Taxonomy of Risks posed by Language Models (2022, background): “excludes, or subsumes those deviating from a norm into ill-fitting categories, affected individuals may also encounter allocational or representational harm [100, 159].”