Neural Machine Translation by Jointly Learning to Align and Translate

By Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. International Conference on Learning Representations, 2014

29,733 citations recorded by the source corpus.

Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Model Architectures

View Paper: https://arxiv.org/abs/1409.0473

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Yoshua Bengio
  4. Percy Liang
  5. Luke S. Zettlemoyer
  6. Quoc V. Le
  7. Graham Neubig
  8. Kyunghyun Cho

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Yi Yang

In Natural Language Processing

  1. Mohit Bansal
  2. Graham Neubig
  3. Jianfeng Gao
  4. Luke S. Zettlemoyer
  5. Kyunghyun Cho
  6. Percy Liang
  7. Yejin Choi
  8. Richard Socher

In Model Architectures

  1. Yoshua Bengio
  2. Quoc V. Le
  3. Kyunghyun Cho
  4. Ruslan Salakhutdinov
  5. Mohit Bansal
  6. Jianfeng Gao
  7. Yoav Goldberg
  8. Richard Socher

Citations in the corpus