ChatGPT outperforms humans in emotional awareness evaluations

By Zohar Elyoseph, D. Hadar-Shoval, K. Asraf, Maya Lvovsky. Frontiers in Psychology, 2023

295 citations recorded by the source corpus.

The artificial intelligence chatbot, ChatGPT, has gained widespread attention for its ability to perform natural language processing tasks and has the fastest-growing user base in history. Although ChatGPT has successfully generated theoretical information in multiple fields, its ability to identify and describe emotions is still unknown. Emotional awareness (EA), the ability to conceptualize one’s own and others’ emotions, is considered a transdiagnostic mechanism for psychopathology. This study utilized the Levels of Emotional Awareness Scale (LEAS) as an objective, performance-based test to analyze ChatGPT’s responses to twenty scenarios and compared its EA performance with that of the general population norms, as reported by a previous study. A second examination was performed one month later to measure EA improvement over time. Finally, two independent licensed psychologists evaluated the fit-to-context of ChatGPT’s EA responses. In the first examination, ChatGPT demonstrated significantly higher performance than the general population on all the LEAS scales (Z score = 2.84). In the second examination, ChatGPT’s performance significantly improved, almost reaching the maximum possible LEAS score (Z score = 4.26). Its accuracy levels were also extremely high (9.7/10). The study demonstrated that ChatGPT can generate appropriate EA responses, and that its performance may improve significantly over time. The study has theoretical and clinical implications, as ChatGPT can be used as part of cognitive training for clinical populations with EA impairments. In addition, ChatGPT’s EA-like abilities may facilitate psychiatric diagnosis and assessment and be used to enhance emotional language. Further research is warranted to better understand the potential benefits and risks of ChatGPT and refine it to promote mental health.

View Paper: https://www.semanticscholar.org/paper/94d96e6bbffa0d4c877f8ba49b706286f6366f2a

Citations in the corpus

• A framework for human evaluation of large language models in healthcare derived from literature review (2024, unknown): “52 usedICCtoassess rater agreement and diagnostic capabilities.”
• The rise and potential of large language model based agents: a survey (2025, unknown): “LLMs can accurately identify user emotions, exhibit empathy [420, 421], and regulate their emotional responses [386,422,423].”
• Large language models empowered agent-based modeling and simulation: a survey and perspectives (2023, unknown): “…and Schulz, 2023; Singh et al., 2023) use psychological tests to simulate the human response to test the cognitive ability, emotional intelligence (Elyoseph et al., 2023), and psychological well-being (Li et al., 2022) of LLMs, demonstrating that LLM agents have human-like intelligence to a…”