Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study

By Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, Daniel Hershcovich. C3NLP, 2023

328 citations recorded by the source corpus.

The recent release of ChatGPT has garnered widespread recognition for its exceptional ability to generate human-like conversations. Given its usage by users from various nations and its training on a vast multilingual corpus that includes diverse cultural and societal norms, it is crucial to evaluate its effectiveness in cultural adaptation. In this paper, we investigate the underlying cultural background of ChatGPT by analyzing its responses to questions designed to quantify human cultural differences. Our findings suggest that, when prompted with American context, ChatGPT exhibits a strong alignment with American culture, but it adapts less effectively to other cultural contexts. Furthermore, by using different prompts to probe the model, we show that English prompts reduce the variance in model responses, flattening out cultural differences and biasing them towards American culture. This study provides valuable insights into the cultural implications of ChatGPT and highlights the necessity of greater diversity and cultural awareness in language technologies.

Subfields: Artificial Intelligence, General Health Professions

View Paper: https://www.semanticscholar.org/paper/ca94c924d8a3b77a2bd5b16ffc03b8723bce9c1f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Meng
2. Chao Zhang
3. Jieyu Zhang
4. Yue Yu
5. Alexander J. Ratner
6. Ranjay Krishna
7. Jindong Wang
8. Xingxu Xie

Citations in the corpus

• Cultural bias and cultural alignment of large language models (2023, background): “LLMs trained on predominantly English text exhibit a latent bias favoring Western cultural values [31, 4], especially when prompted in English [9].”
• Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias (2023, background): “This regional bias exhibited in the generated dataset can pose substantial challenges when constructing reliable machine learning models [23, 6].”
• A Survey on Evaluation of Large Language Models (2023, background): “, 2013); ChatGPT was also observed to exhibit somewhat bias on cultural values (Cao et al., 2023).”