A Recursive Recurrent Neural Network for Statistical Machine Translation

By Shujie Liu, Nan Yang, Mu Li, M. Zhou. 2014

165 citations recorded by the source corpus.

In this paper, we propose a novel recursive recurrent neural network (R 2 NN) to model the end-to-end decoding process for statistical machine translation. R 2 NN is a combination of recursive neural network and recurrent neural network, and in turn integrates their respective capabilities: (1) new information can be used to generate the next hidden state, like recurrent neural networks, so that language model and translation model can be integrated naturally; (2) a tree structure can be built, as recursive neural networks, so as to generate the translation candidates in a bottom up manner. A semi-supervised training approach is proposed to train the parameters, and the phrase pair embedding is explored to model translation confidence directly. Experiments on a Chinese to English translation task show that our proposed R 2 NN can outperform the stateof-the-art baseline by about 1.5 points in BLEU.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “This template is naturally suited for many NLP tasks such as language modeling [2, 64, 65], machine translation [66, 67, 68], speech recognition [69, 70, 71, 72], image captioning [73].”
Not All Contexts Are Created Equal: Better Word Representations with Variable Attention (2015, background): “…raw text data have been shown to improve many NLP tasks, such as part-of-speech tagging (Collobert et al., 2011), dependency parsing (Chen and Manning, 2014; Kong et al., 2014) and machine translation (Liu et al., 2014; Kalchbrenner and Blunsom, 2013; Devlin et al., 2014; Sutskever et al., 2014).”
Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation (2015, background): “Learning vector representations of words by treating them as optimizable parameters in various kinds of language models has been found to be a remarkably effective means for generating vector representations that perform well in other tasks (Collobert et al., 2011; Kalchbrenner and Blunsom, 2013; Li”
Speech emotion recognition using convolutional and Recurrent Neural Networks (2016, methodology): “Therefore, these sequential data analysis approaches are widely used not only for language modeling but also for machine translation [20-24].”
Two/Too Simple Adaptations of Word2Vec for Syntax Problems (2015, background): “…models have been shown to improve many NLP tasks, such as part-of-speech tagging (Collobert et al., 2011), dependency parsing (Chen and Man-ning, 2014; Kong et al., 2014) and machine translation (Liu et al., 2014; Kalchbrenner and Blunsom, 2013; Devlin et al., 2014; Sutskever et al., 2014).”
Multi-Task Learning for Multiple Language Translation (2015, background): “Much work has been done to address the data sparsity problem such as the pivot language approach (Wu and Wang, 2007; Cohn and Lapata, 2007) and deep learning techniques (Devlin et al., 2014; Gao et al., 2014; Sundermeyer et al., 2014; Liu et al., 2014).”
Distributed under Creative Commons Cc-by 4.0 Mlitb: Machine Learning in the Browser (background): “…models for many tasks, including computer vision (Krizhevsky, Sutskever & Hinton, 2012; Lin, Chen & Yan, 2013), speech recognition (Hinton et al., 2012), and natural language processing and machine translation (Liu et al., 2014; Bahdanau, Cho & Bengio, 2014; Sutskever, Vinyals & Le, 2014).”
Mercury: Metro density prediction with recurrent neural network on streaming CDR data (2016, unknown): “In [11], Liu et al.”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “This template is naturally suited for many NLP tasks such as language modeling [2], [63], [64], machine translation [65]–[67], speech recognition [68]–[71], image captioning [72].”
Hierarchical Recurrent Neural Network for Document Modeling (2015, background): “Liu et al. (2014) propose a recursive recurrent neural network (R 2 NN) for end-to-end decoding to help improve translation quality.”
Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts (2016, background): “Recurrent Neural Networks (RNNs) have shown great promise in machine translation tasks(Liu et al., 2014; Sutskever et al., 2014; Auli et al., 2013).”