Improving SMT quality with morpho-syntactic analysis

By Sonja Nießen, Hermann Ney. 2000

103 citations recorded by the source corpus.

In the framework of statistical machine translation (SMT), correspondences between the words in the source and the target language are learned from bilingual corpora on the basis of so-called alignment models. Many of the statistical systems use little or no linguistic knowledge to structure the underlying models. In this paper we argue that training data is typically not large enough to sufficiently represent the range of different phenomena in natural languages and that SMT can take advantage of the explicit introduction of some knowledge about the languages under consideration. The improvement of the translation results is demonstrated on two different German-English corpora.

Subfields: Artificial Intelligence, Natural Language Processing, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Machine Translation

Citations in the corpus

What do Neural Machine Translation Models Learn about Morphology? (2017, methodology): “morphologically-rich languages resort to various techniques for representing morphological knowledge, such as word segmentation (Nieflen and Ney, 2000; Koehn and Knight, 2003; Badr et al., 2008) and factored translation and reordering”
How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs (2016, neutral): “We report results for a number of systems, and find that recently introduced character-level NMT systems perform better at transliteration than models with byte-pair encoding (BPE) segmentation, but perform more poorly at morphosyntactic agreement, and translating discontiguous units of meaning.”
Neural Machine Translation with Byte-Level Subwords (2019, methodology): “Related Work Subword Vocabularies Previous works have shown that finer-grained vocabularies consistently outperforms wordlevel vocabularies in many settings, for example, vocabularies based on morpheme segmentation (Nießen and Ney 2000; Luong, Socher, and Manning 2013), byte-pair encoding (Sennrich,”
A Joint Dependency Model of Morphological and Syntactic Structure for Statistical Machine Translation (2015, background): “The main focus of research on compound splitting has been on the splitting algorithm (Popovic et al., 2006; Nießen and Ney, 2000; Weller et al., 2014; Macherey et al., 2011).”
Adaptation of machine translation for multilingual information retrieval in the medical domain (2014, result): “Language models of order 5 are estimated using SRILM (Stanford Research Institute Language Modeling toolkit) [83] with modified Kneser- Ney smoothing [84] is used in decoding for querying the models .”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Super-word levels texts utterances sentences phrases relations words morphems syllables characters phonemes Sub-word levels been shown to improve machine translation performance [6, 15, 139, 184, 258].”
Machine Translation without Words through Substring Alignment (2012, background): “…works have attempted to handle morphology, decompounding and regularization through lemmatization, morphological analysis, or unsupervised techniques (Nießen and Ney, 2000; Brown, 2002; Lee, 2004; Goldwater and McClosky, 2005; Talbot and Osborne, 2006; Mermer and Akın, 2010; Macherey et al., 2011).”