It’s not a Non-Issue: Negation as a Source of Error in Machine Translation

By Md Mosharaf Hossain, Antonios Anastasopoulos, Eduardo Blanco, Alexis Palmer. 2020

32 citations recorded by the source corpus.

As machine translation (MT) systems progress at a rapid pace, questions of their adequacy linger. In this study we focus on negation, a universal, core property of human language that significantly affects the semantics of an utterance. We investigate whether translating negation is an issue for modern MT systems using 17 translation directions as test bed. Through thorough analysis, we find that indeed the presence of negation can significantly impact downstream quality, in some cases resulting in quality reductions of more than 60%. We also provide a linguistically motivated analysis that directly explains the majority of our findings. We release our annotations and code to replicate our analysis here: https://github.com/mosharafhossain/negation-mt.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

AI and the Everything in the Whole Wide World Benchmark (2021, background): “Metrics for MT systems (including but not limited to BLEU) are also not good at measuring the impact of negation errors [Hossain et al., 2020].”
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation (2022, background): ““We ﬁnd that indeed the presence of negation can signiﬁcantly impact downstream quality [of machine translation systems]” (Hossain et al., 2020a).”