Speech recognition with deep recurrent neural networks

By Alex Graves, Abdel-rahman Mohamed, Geoffrey E. Hinton. 2013

9,035 citations recorded by the source corpus.

Recurrent neural networks (RNNs) are a powerful model for sequential data. End-to-end training methods such as Connectionist Temporal Classification make it possible to train RNNs for sequence labelling problems where the input-output alignment is unknown. The combination of these methods with the Long Short-term Memory RNN architecture has proved particularly fruitful, delivering state-of-the-art results in cursive handwriting recognition. However RNN performance in speech recognition has so far been disappointing, with better results returned by deep feedforward networks. This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs. When trained end-to-end with suitable regularisation, we find that deep Long Short-term Memory RNNs achieve a test set error of 17.7% on the TIMIT phoneme recognition benchmark, which to our knowledge is the best recorded score.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Mohit Bansal
  3. Quoc V. Le
  4. Jianfeng Gao
  5. Percy Liang
  6. Sergey Levine
  7. Kyunghyun Cho
  8. Ruslan Salakhutdinov

In Natural Language Processing

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Graham Neubig
  4. Kyunghyun Cho
  5. Yoshua Bengio
  6. Richard Socher
  7. Percy Liang
  8. Luke S. Zettlemoyer

In Signal Processing

  1. Jinyu Li
  2. Dong Yu
  3. Yoshua Bengio
  4. Shinji Watanabe
  5. Li Deng
  6. Yu Zhang
  7. Tara N. Sainath
  8. James R. Glass

Citations in the corpus