Mirages: On Anthropomorphism in Dialogue Systems

By Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Zeerak Talat. 2023

69 citations recorded by the source corpus.

Automated dialogue or conversational systems are anthropomorphised by developers and personified by users. While a degree of anthropomorphism may be inevitable due to the choice of medium, conscious and unconscious design choices can guide users to personify such systems to varying degrees. Encouraging users to relate to automated systems as if they were human can lead to high risk scenarios caused by over-reliance on their outputs. As a result, natural language processing researchers have investigated the factors that induce personification and develop resources to mitigate such effects. However, these efforts are fragmented, and many aspects of anthropomorphism have yet to be explored. In this paper, we discuss the linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise, including reinforcing gender stereotypes and notions of acceptable language. We recommend that future efforts towards developing dialogue systems take particular care in their design, development, release, and description; and attend to the many linguistic cues that can elicit personification by users.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “The study of human-generative AI relationships is nascent, but growing, and highlights that the anthropomorphism [13] of these technologies may contribute to unfounded trust and reliance [192, 225].”
Generative Agents: Interactive Simulacra of Human Behavior (2023, background): “While this tendencymay increase user engagement, it also poses risks, such as users becoming overly reliant on or emotionally attached to the agents [1].”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “…corresponding expectations (Nass and Moon, 2000), and even subtle language and communication cues can have profound impact on people’s mental model (Abercrombie et al., 2023). p Effective approaches to trans-parency should take into account the receivers’ existing perception of what the model or…”