Linguistically Motivated Vocabulary Reduction for Neural Machine Translation from Turkish to English

By Duygu Ataman, Matteo Negri, Marco Turchi, Marcello Federico. 2017

122 citations recorded by the source corpus.

Abstract The necessity of using a fixed-size word vocabulary in order to control the model complexity in state-of-the-art neural machine translation (NMT) systems is an important bottleneck on performance, especially for morphologically rich languages. Conventional methods that aim to overcome this problem by using sub-word or character-level representations solely rely on statistics and disregard the linguistic properties of words, which leads to interruptions in the word structure and causes semantic and syntactic losses. In this paper, we propose a new vocabulary reduction method for NMT, which can reduce the vocabulary of a given input corpus at any rate while also considering the morphological properties of the language. Our method is based on unsupervised morphology learning and can be, in principle, used for pre-processing any language pair. We also present an alternative word segmentation method based on supervised morphological analysis, which aids us in measuring the accuracy of our model. We evaluate our method in Turkish-to-English NMT task where the input language is morphologically rich and agglutinative. We analyze different representation methods in terms of translation accuracy as well as the semantic and syntactic properties of the generated output. Our method obtains a significant improvement of 2.3 BLEU points over the conventional vocabulary reduction technique, showing that it can provide better accuracy in open vocabulary translation of morphologically rich languages.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Neural Machine Translation for English–Kazakh with Morphological Segmentation and Synthetic Data (2019, methodology): “Unsupervised morphological segmentation is performed with LMVR (Ataman et al., 2017), 6 a variant of Morfessor (Virpioja et al., 2013) that allows a ﬁxed vocabulary size to be deﬁned.”
Finding Better Subword Segmentation for Neural Machine Translation (2018, background): “Some other work segmented words based on morphological information to obtain the representation of OOV words [2,5], which can be called linguistically motivated method.”
Modeling Target-Side Inflection in Neural Machine Translation (2017, methodology): “Finally, there has been further more recent work on alternatives to using BPE segmentation for NMT. Ataman et al. (2017) looked at segmentation for Turkish, which is an agglutinative language.”
Gender Bias in Machine Translation (2021, background): “For instance, current systems segment text into subword units with statistical methods that can break the morphological structure of words, thus losing relevant semantic and syntactic information in morphologically-rich languages (Niehues et al., 2016; Ataman et al., 2017).”
Neural Machine Translation: A Review (2019, background): “However, linguistically-motivated subword units (Ataman et al., 2017; Huck et al., 2017; Macháček et al., 2018; Pinnis et al., 2017) that also take morpheme boundaries into account do not always improve over completely data-driven ones.”