Translation Modeling with Bidirectional Recurrent Neural Networks

By Martin Sundermeyer, Tamer Alkhouli, Joern Wuebker, Hermann Ney. 2014

195 citations recorded by the source corpus.

This work presents two different translation models using recurrent neural networks. The first one is a word-based approach using word alignments. Second, we present phrase-based translation models that are more consistent with phrasebased decoding. Moreover, we introduce bidirectional recurrent neural models to the problem of machine translation, allowing us to use the full source sentence in our models, which is also of theoretical interest. We demonstrate that our translation models are capable of improving strong baselines already including recurrent neural language models on three tasks: IWSLT 2013 German!English, BOLT Arabic!English and Chinese!English. We obtain gains up to 1.6% BLEU and 1.7% TER by rescoring 1000-best lists.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “An important point that they mentioned was the applicability of their conclusions to a variety of other tasks such as statistical machine translation (Sundermeyer et al., 2014).”
Sequence-to-sequence neural net models for grapheme-to-phoneme conversion (2015, methodology): “Bi-directional LSTMs have been applied to speech recognition [17] and machine translation [6].”
Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking (2015, background): “Previously, bidirectional RNNs (Schuster and Paliwal, 1997) have been shown to be effective for handwriting recognition (Graves et al., 2008), speech recognition (Graves et al., 2013), and machine translation (Sundermeyer et al., 2014).”
Text normalization with convolutional neural networks (2018, background): “Sequence-to-sequence (seq2seq) learning has shown great results in many application scenarios, such as speech recognition, text-to-speech synthesis and machine translation (El-Desoky and Schuller 2016; Lu et al. 2016; Sundermeyer et al. 2014; Sutskever et al. 2014).”
Combined spoken language translation (2014, methodology): “Additionally to the standard system combination pipeline presented in [1, 2], we applied a recurrent neural network rescoring step [3] for the English→French MT task.”
From Feedforward to Recurrent LSTM Neural Networks for Language Modeling (2015, background): “Nevertheless , the conclusions we draw are not limited to speech recognition , but can be generalized to other problems like statistical machine translation, as shown e.g. in [13].”
Applying deep learning to answer selection: A study and an open task (2015, background): “on image recognition [9], on speech [10] [11] [12] and on machine translation [13] [14].”
Writing Stories with Help from Recurrent Neural Networks (2016, methodology): “Early approaches used hand-authored formal models of a particular story-world domain to generate narratives pertaining to that domain (Klein, Aeschli-mann, and Balsiger 1973; Lebowitz 1985; Meehan 1977).”
FactorNet: a deep learning framework for predicting cell type specific transcription factor binding from nucleotide-resolution sequential data (2017, background): “, 2013), machine 60 translation (Sundermeyer et al., 2014), and human action recognition (Zhu et al.”
De-identification of patient notes with recurrent neural networks (2016, background): “…2014; Lee and Dernoncourt, 2016), question answering (We-ston et al., 2015; Wang and Nyberg, 2015), machine translation (Bahdanau et al., 2014; Tamura et al., 2014; Sundermeyer et al., 2014), as well as named entity recognition (Collobert et al., 2011; Lample et al., 2016; Labeau et al., 2015).”
Quaddirectional 2D-Recurrent Neural Networks For Image Labeling (2015, background): “On the other hand, Recurrent Neural Networks (RNN)have achieved extensive successes in 1-D sequence modeling [25]–[27].”
The RWTH Aachen University English-Romanian Machine Translation System for WMT 2016 (2016, methodology): “For more details we refer the reader to (Sundermeyer et al., 2014a).”
DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences (2015, background): “BLSTMs can capture long-term dependencies and have been effective for other machine learning applications such as phoneme classification (10), speech recognition (11), machine translation (12) and human action recognition (13).”
Neural Networks and Deep Learning (2018, unknown): “Bidirectional recurrent networks and LSTMs for protein structure prediction, handwriting recognition, translation, and speech recognition are discussed in [20, 154, 155, 157, 378, 477].”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “An important point that they mentioned was the applicability of their conclusions to a variety of other tasks such as statistical machine translation [88].”
Multi-Task Learning for Multiple Language Translation (2015, background): “Much work has been done to address the data sparsity problem such as the pivot language approach (Wu and Wang, 2007; Cohn and Lapata, 2007) and deep learning techniques (Devlin et al., 2014; Gao et al., 2014; Sundermeyer et al., 2014; Liu et al., 2014).”