“You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases

By Dirk Hovy, Federico Bianchi, Tommaso Fornaciari. 2020

60 citations recorded by the source corpus.

The main goal of machine translation has been to convey the correct content. Stylistic considerations have been at best secondary. We show that as a consequence, the output of three commercial machine translation systems (Bing, DeepL, Google) make demographically diverse samples from five languages “sound” older and more male than the original. Our findings suggest that translation models reflect demographic bias in the training data. This opens up interesting new research avenues in machine translation to take stylistic considerations into account.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kai-Wei Chang
2. Nanyun Peng
3. Yejin Choi
4. Eduard H. Hovy
5. Diyi Yang
6. Dirk Hovy
7. Emily Sheng
8. Percy Liang

In Natural Language Processing

1. Kai-Wei Chang
2. Nanyun Peng
3. Yejin Choi
4. Emily Sheng
5. Diyi Yang
6. Eduard H. Hovy
7. Dirk Hovy
8. Hal Daumé Iii

Citations in the corpus

• Societal Biases in Language Generation: Progress and Challenges (2021, background): “, 2021) MT (Vanmassenhove et al., 2018; Elaraby et al., 2018; Prates et al., 2019; Stanovsky et al., 2019; Escudé Font and Costa-jussà, 2019; Cho et al., 2019; Moryossef et al., 2019; Saunders and Byrne, 2020; Saunders et al., 2020; Kocmi et al., 2020; Costa-jussà and de Jorge, 2020; Costa-jussà et ”
• Multi-Dimensional Gender Bias Classification (2020, neutral): “, 2019), and machine translation (Stanovsky et al., 2019; Hovy et al., 2020).”
• Five sources of bias in natural language processing (2021, background): “Hovy et al. (2020) found that machine translation systems changed the perceived user demographics to make samples sound older and more male in translation.”
• Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications (2022, background): “…(Novikova et al., 2017; Howcroft et al., 2020; Liang and Li, 2021), including examinations of how their design, deployment, and evaluation can give rise to ethical concerns (Cercas Curry and Rieser, 2018; Schlesinger et al., 2018; Hovy et al., 2020; Sheng et al., 2021; Abercrombie et al., 2021).”
• The Importance of Modeling Social Factors of Language: Theory and Practice (2021, background): “or overestimating one particular group’s attributes can skew the results, for example, in the case of machine-translated texts sounding older and more male (Hovy et al., 2020).”
• Gender Bias in Machine Translation (2021, background): “(Hovy et al., 2020) Trustpilot (reviews with gender and age) b R: under-rep”