Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation
By Rico Sennrich. 2015
33 citations recorded by the source corpus.
The role of language models in SMT is to promote fluent translation output, but traditional n-gram language models are unable to capture fluency phenomena between distant words, such as some morphological agreement phenomena, subcategorisation, and syntactic collocations with string-level gaps. Syntactic language models have the potential to fill this modelling gap. We propose a language model for dependency structures that is relational rather than configurational and thus particularly suited for languages with a (relatively) free word order. It is trainable with Neural Networks, and not only improves over standard n-gram language models, but also outperforms related syntactic language models. We empirically demonstrate its effectiveness in terms of perplexity and as a feature function in string-to-tree SMT from English to German and Russian. We also show that using a syntactic evaluation metric to tune the log-linear parameters of an SMT system further increases translation quality when coupled with a syntactic language model.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Philipp Koehn
- Felix Stahlberg
- Rico Sennrich
- Barry Haddow
- Ondrej Bojar
- Marcin Junczys-Dowmunt
- Graham Neubig
- Yonatan Belinkov
Citations in the corpus
- Getting Gender Right in Neural Machine Translation (2019, methodology): “In order to by-pass the OOV problem and reduce the number of dictionary entries, we use word-segmentation with BPE (Sennrich, 2015).”
- A Joint Dependency Model of Morphological and Syntactic Structure for Statistical Machine Translation (2015, methodology): “Additionally, our baseline system contains a dependency language model (RDLM) (Sennrich, 2015), trained on the target-side of the parallel training data.”
- Syntax-aware Neural Machine Translation Using CCG (2017, background): “Syntax has helped in statistical machine translation (SMT) to capture dependencies between distant words that impact morphological agreement, subcategorisation and word order (Galley et al., 2004; Menezes and Quirk, 2007; Williams and Koehn, 2012; Sennrich, 2015; Chiang, 2007).”
- Predicting Target Language CCG Supertags Improves Neural Machine Translation (2017, background): “capture dependencies between distant words that impact morphological agreement, subcategorisation and word order (Galley et al.,2004;Menezes and Quirk,2007;Williams and Koehn,2012;Nadejde et al.,2013;Sennrich, 2015;Nadejde et al.,2016a,b;Chiang,2007). There has been some work in NMT on modeling sour”
- Edinburgh’s Statistical Machine Translation Systems for WMT16 (2016, methodology): “The parameters of the model are tuned towards the linear interpolation of B LEU and the syntactic metric HWCM (Liu and Gildea, 2005; Sennrich, 2015).”