Socially Aware Bias Measurements for Hindi Language Representations

By Vijit Malik, Sunipa Dev, A. Nishi, Nanyun Peng, Kai-Wei Chang. 2021

50 citations recorded by the source corpus.

Language representations are an efficient tool used across NLP, but they are strife with encoded societal biases. These biases are studied extensively, but with a primary focus on English language representations and biases common in the context of Western society. In this work, we investigate the biases present in Hindi language representations such as caste and religion associated biases. We demonstrate how biases are unique to specific language representations based on the history and culture of the region they are widely spoken in, and also how the same societal bias (such as binary gender associated biases) when investigated across languages is encoded by different words and text spans. With this work, we emphasize on the necessity of social-awareness along with linguistic and grammatical artefacts when modeling language representations, in order to understand the biases encoded.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “For certain modalities, such as language, biases can be represented differently [142].”
Re-contextualizing Fairness in NLP: The Case of India (2022, methodology): “along the lines of recent work on Hindi, Bengali, and Telugu (Malik et al., 2021; Pujari et al., 2019).”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “In an Indian context, for example, even those who identify as belonging to the “same” caste (Malik et al., 2021), can have varied lived experiences based on class, gender, and other identities.”
Undesirable biases in NLP: Averting a crisis of measurement (2022, methodology): “) In Hindi, no accurate translations of Shakespeare and NASA exist (Malik et al., 2022).”
Efficient Gender Debiasing of Pre-trained Indic Language Models (2022, background): “Malik et al. (2022) quantiﬁed social biases pertinent to the Hindi language, like caste and religion, along with gender bias.”
Bias and Fairness in Large Language Models: A Survey (2023, background): “Meade et al. (2023) compare generated outputs to safe example responses from similar contexts, reranking candidate responses based on their similarity to the safe example.”