Evaluating Gender Bias in Hindi-English Machine Translation

By Gauri Gupta, Krithika Ramesh, Sanjay Singh. 2021

32 citations recorded by the source corpus.

With language models being deployed increasingly in the real world, it is essential to address the issue of the fairness of their outputs. The word embedding representations of these language models often implicitly draw unwanted associations that form a social bias within the model. The nature of gendered languages like Hindi, poses an additional problem to the quantification and mitigation of bias, owing to the change in the form of the words in the sentence, based on the gender of the subject. Additionally, there is sparse work done in the realm of measuring and debiasing systems for Indic languages. In our work, we attempt to evaluate and quantify the gender bias within a Hindi-English machine translation system. We implement a modified version of the existing TGBI metric based on the grammatical considerations for Hindi. We also compare and contrast the resulting bias measurements across multiple metrics for pre-trained embeddings and the ones learned by our machine translation model.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Re-contextualizing Fairness in NLP: The Case of India (2022, background): “2 There is commendable recent work on fairness in NLP models for Indian languages such as Hindi, Bengali, and Telugu (Pujari et al., 2019; Malik et al., 2021; Gupta et al., 2021).”
Efficient Gender Debiasing of Pre-trained Indic Language Models (2022, background): “More recent work by Ramesh et al. (2021) quantiﬁed bias in English-Hindi machine translation.”
Theories of “Gender” in NLP Bias Research (2022, background): “12801 [113] Krithika Ramesh, Gauri Gupta, and Sanjay Singh.”