Learning Semantic Representations for the Phrase Translation Model

By Jianfeng Gao, Xiaodong He, Wen-tau Yih, Li Deng. 2013

32 citations recorded by the source corpus.

This paper presents a novel semantic-based phrase translation model. A pair of source and target phrases are projected into continuous-valued vector representations in a low-dimensional latent semantic space, where their translation score is computed by the distance between the pair in this new space. The projection is performed by a multi-layer neural network whose weights are learned on parallel training data. The learning is aimed to directly optimize the quality of end-to-end machine translation results. Experimental evaluation has been performed on two Europarl translation tasks, English-French and German-English. The results show that the new semantic-based phrase translation model significantly improves the performance of a state-of-the-art phrase-based statistical machine translation sys-tem, leading to a gain of 0.7-1.0 BLEU points.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (2014, methodology): “A similar approach of using bag-of-words representations was proposed in (Gao et al., 2013) as well.”
Text Understanding from Scratch (2015, background): “, 2013)(Gao et al., 2013)(Le & Mikolov, 2014)(Mikolov et al.”
Deep Learning: Methods and Applications (2014, background): “A more recent study on applying deep learning methods to machine translation appears in [121, 123], where the phrase-translation component, rather than the LM component in the machine translation system is replaced by the neural network models with semantic word embeddings.”
Joint Language and Translation Modeling with Recurrent Neural Networks (2013, neutral): “Our approach of combining language and translation modeling is very much in line with recent work on n-gram-based translation models (Crego and Yvon, 2010), and more recently continuous space-based translation models (Le et al., 2012a; Gao et al., 2013).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “The concept of word embedding was very recently extended from a single language to two, producing bilingual word embeddings for machine translation applications (Zou et al., 2013; Gao et al., 2013).”
A Neural Probabilistic Model for Context Based Citation Recommendation (2015, background): “…such as parsing (Collobert and Weston 2008; Socher et al. 2013a), sentiment analysis (Socher et al. 2013b), language modeling (Bengio et al. 2003; Mnih and Hinton 2007; Mikolov et al. 2010; Mnih and Teh 2012; Huang et al. 2012a; Mikolov et al. 2013), and machine translation (Gao et al. 2014).”
Bilingually-constrained Phrase Embeddings for Machine Translation (2014, background): “Gao et al. (2013) also use bag-of-words but learn BLEU sensitive phrase embed-dings.”
An Autoencoder Approach to Learning Bilingual Word Representations (2014, methodology): “Looking more generally at neural networks that learn multilingual representations of words or phrases, we mention the work of Gao et al. (2013) which showed that a useful linear mapping between separately trained monolingual skip-gram language models could be learned.”