Application of Pretrained Deep Neural Networks to Large Vocabulary Speech Recognition

By Navdeep Jaitly, Patrick Nguyen, Andrew W. Senior, Vincent Vanhoucke. 2012

262 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Yoshua Bengio
5. Quoc V. Le
6. Tara N. Sainath
7. James R. Glass
8. Jianfeng Gao

In Model Architectures

1. Yoshua Bengio
2. Quoc V. Le
3. Dong Yu
4. Jinyu Li
5. Tara N. Sainath
6. Li Deng
7. Oriol Vinyals
8. Navdeep Jaitly

In Speech & Audio

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Yu Zhang
6. James R. Glass
7. Shinji Watanabe
8. Wei-Ning Hsu

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Li Deng
4. Tara N. Sainath
5. Yoshua Bengio
6. Shinji Watanabe
7. Yifan Gong
8. Yu Zhang

Citations in the corpus

• ADADELTA: An Adaptive Learning Rate Method (2012, methodology): “The neural network is setup as in [7] where the inputs are 26 frames of audio, each consisting of 40 log-energy filter bank outputs.”
• Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, background): “More recently DNNs have become a fixture in the ASR pipeline with almost all state of the art speech work containing some form of deep neural network [42, 29, 17, 16, 43, 58].”
• Listen, attend and spell: A neural network for large vocabulary conversational speech recognition (2015, background): “ntroduction Deep Neural Networks (DNNs) have led to improvements in various components of speech recognizers. They are commonly used in hybrid DNN-HMM speech recognition systems for acoustic modeling [1, 2, 3, 4, 5, 6]. DNNs have also produced significant gains in pronunciation models that map words ”
• Learning Factored Representations in a Deep Mixture of Experts (2013, methodology): “This dataset is a random subset of approximately one million samples from a larger proprietary database of several hundred hours of US English data collected using Voice Search, Voice Typing and read data [8].”
• Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition (2014, methodology): “While DNNs have shown state of the art performance in both phone recognition and large vocabulary speech recognition [9, 10, 11], the application of LSTM networks has been limited to phone recognition on the TIMIT database, and it has required using additional techniques and models such as CTC and R”
• Indoor Semantic Segmentation using depth information (2013, background): “ning approaches are particularly adapted to the addition of new image modalities such as depth information. Its recent success for dealing with various types of data is manifest in speech recognition [13], molecular activity prediction, object recognition [12] and many 1 Performed the work at New Yo”
• A tutorial survey of architectures, algorithms, and applications for deep learning (2014, neutral): “, 2012), Google (Jaitly et al., 2012; Dean et al., 2012), and Baidu.”
• Factorized Hidden Layer Adaptation for Deep Neural Network Based Acoustic Modeling (2016, methodology): “I. INTRODUCTION D EEP neural network (DNN) based acoustic modelinghas significantly outperformed the conventional Gaussian mixture model (GMM) based automatic speech recognition (ASR) systems [1]–[5].”
• Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition (2014, background): “N Recently, the context dependent (CD) deep neural network (DNN) hidden Markov model (HMM) (CD-DNN-HMM) has become the dominant framework for acoustic modeling in speech recognition (e.g. [1][2][3][4][5]). However, given that speech is an inherently dynamic process, some researchers pointed out that”
• Speaker adaptation of context dependent deep neural networks (2013, methodology): “The first DNN deployed in production by Google [15] has the topology shown in figure 1.”
• Long short-term memory recurrent neural network architectures for large scale acoustic modeling (2014, methodology): “In terms of architectures, following the success of DNNs for acoustic modeling [13, 14, 15, 16], a deep BLSTM RNN combined with a CTC output layer and an RNN transducer predicting phone sequences has been shown to reach state-of-the-art phone recognition accuracy on the TIMIT database [17].”
• Towards End-To-End Speech Recognition with Recurrent Neural Networks (2014, methodology): “Full-sequence training methods such as Maximum Mutual Information have been used to directly train HMM-neural network hybrids to maximise the probability of the correct transcription (Bahl et al., 1986; Jaitly et al., 2012).”
• Multilingual acoustic models using distributed deep neural networks (2013, methodology): “The setup for DNN training and the hybrid decoding is based on the setup in [31].”
• Accurate and compact large vocabulary speech recognition on mobile devices (2013, methodology): “We have previously described the use of deep neural networks for probability estimation in our cloud-based mobile voice recognition system [9].”
• Acoustic Modeling With Hierarchical Reservoirs (2013, unknown): “lary Continuous Speech Recognition (LVCSR) [9]–[14].”