On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

By Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio. 2014

7,554 citations recorded by the source corpus.

Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Yoshua Bengio
  3. Jianfeng Gao
  4. Percy Liang
  5. Graham Neubig
  6. Luke S. Zettlemoyer
  7. Quoc V. Le
  8. Kyunghyun Cho

In Molecular Biology

  1. Matthias Fey
  2. Trevor Cohn
  3. David Chiang
  4. Tommi S. Jaakkola
  5. Kyunghyun Cho
  6. Yoshua Bengio
  7. Jonathan Berant
  8. Jure Leskovec

In Natural Language Processing

  1. Mohit Bansal
  2. Graham Neubig
  3. Kyunghyun Cho
  4. Jianfeng Gao
  5. Luke S. Zettlemoyer
  6. Yejin Choi
  7. Noah A. Smith
  8. Percy Liang

In Model Architectures

  1. Yoshua Bengio
  2. Quoc V. Le
  3. Kyunghyun Cho
  4. Yoav Goldberg
  5. Ruslan Salakhutdinov
  6. Yonatan Belinkov
  7. Richard Socher
  8. Graham Neubig

Citations in the corpus