Mitigating Gender Bias in Machine Translation with Target Gender Annotations

By Toms Bergmanis, Arturs Stafanovivcs, Marcis Pinnis. 2020

69 citations recorded by the source corpus.

When translating The secretary asked for details. to a language with grammatical gender, it might be necessary to determine the gender of the subject secretary. If the sentence does not contain the necessary information, it is not always possible to disambiguate. In such cases, machine translation systems select the most common translation option, which often corresponds to the stereotypical translations, thus potentially exacerbating prejudice and marginalisation of certain groups and people. We argue that the information necessary for an adequate translation can not always be deduced from the sentence being translated or even might depend on external knowledge. Therefore, in this work, we propose to decouple the task of acquiring the necessary information from the task of learning to translate correctly when such information is available. To that end, we present a method for training machine translation systems to use word-level annotations containing information about subject’s gender. To prepare training data, we annotate regular source language words with grammatical gender information of the corresponding target language words. Using such data to train machine translation systems reduces their reliance on gender stereotypes when information about the subject’s gender is available. Our experiments on five language pairs show that this allows improving accuracy on the WinoMT test set by up to 25.8 percentage points.

Subfields: Artificial Intelligence, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “, 2020a) as well as machine translation (Saunders and Byrne, 2020; Costa-jussà and de Jorge, 2020; Stafanovičs et al., 2020).”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Augmentation Discrimination Adding synthetically generated data (Dinan et al., 2020; Liu et al., 2020; Stafanovičs et al., 2020) Toxicity Adding safer example data (Mathew et al.”
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender (2021, background): “Several works aim to enrich the gender input to an MT system by adding additional gold annotation or context (Stafanovičs et al., 2020; Saunders et al., 2020; Moryossef et al., 2019).”
A Survey on Gender Bias in Natural Language Processing (2021, methodology): “WinoMT has become widely applied as a challenge dataset for gender bias detection in MT systems [Basta et al. 2020; Renduchintala et al. 2021; Saunders and Byrne 2020; Stafanovičs et al. 2020] with Saunders et al.”
Gender-Aware Reinflection using Linguistically Enhanced Neural Models (2020, background): “…translation (Rabinovich et al., 2017; Vanmassenhove et al., 2018; Font and Costa-jussà, 2019; Moryossef et al., 2019; Stanovsky et al., 2019; Stafanoviˇcs et al., 2020; Gonen and Webster, 2020), named entity recognition (Mehrabi et al., 2019), dialogue systems (Dinan et al., 2019), and…”
Improving Gender Translation Accuracy with Filtered Self-Training (2021, unknown): “biguous input or propose methods to control for gender or to augment data with gender information (Elaraby et al., 2018; Moryossef et al., 2019; Prates et al., 2018; Saunders et al., 2020; Stafanovičs et al., 2020; Vanmassenhove et al., 2018).”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, background): “…group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé III, 2020).”
First the Worst: Finding Better Gender Translations During Beam Search (2021, methodology): “The current dominant approaches to the problem, which involve model retraining (Vanmassenhove et al., 2018; Escudé Font and Costa-jussà, 2019; Stafanovičs et al., 2020) or tuning on targeted gendered bilingual data (Saunders and Byrne, 2020;”