Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

By Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, Yoshua Bengio. 2014

14,897 citations recorded by the source corpus.

In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.

Subfields: Artificial Intelligence, Model Architectures, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Yoshua Bengio
  3. Jianfeng Gao
  4. Percy Liang
  5. Quoc V. Le
  6. Luke S. Zettlemoyer
  7. Sergey Levine
  8. Kyunghyun Cho

In Model Architectures

  1. Yoshua Bengio
  2. Quoc V. Le
  3. Kyunghyun Cho
  4. Richard Socher
  5. Ruslan Salakhutdinov
  6. Yoav Goldberg
  7. Mohit Bansal
  8. Jianfeng Gao

In Signal Processing

  1. Yoshua Bengio
  2. Jinyu Li
  3. Dong Yu
  4. Hung-yi Lee
  5. Shinji Watanabe
  6. Junichi Yamagishi
  7. Yu Zhang
  8. James R. Glass

Citations in the corpus