Deep Belief Networks for phone recognition

By Abdel-rahman Mohamed, George E. Dahl, Geoffrey E. Hinton. 2009

475 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Speech & Audio

Citations in the corpus

Robust Speech Recognition via Large-Scale Weak Supervision (2022, background): “Scaling Speech Recognition Early work applying deep learning to speech recognition found improved performance with model depth and size and leveraged GPU acceleration to make training larger models tractable (Mohamed et al., 2009).”
Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019 (2019, methodology): “Equation 23 shows the energy of a configuration (in matrix notation) of the standard type of RBM that has binary-valued hidden and visible units (a denotes bias weights (offsets) for the visible units, b denotes bias weights for the hidden units,W denotes matrix weight of the connection between hidd”
Deep learning in neural networks: An overview (2014, background): “7% on the TIMIT core test set (Mohamed et al., 2009; Mohamed and Hinton, 2010); compare further improvements through FNNs (Hinton et al.”
Deep Learning: Methods and Applications (2014, methodology): “An early DNN–HMM architecture [257] was presented at the NIPS Workshop [109], developed, analyzed, and assisted by University of Toronto and MSR speech researchers.”
Deep Learning using Linear Support Vector Machines (2013, background): “These include (but not limited to) speech (Mohamed et al., 2009; Dahl et al., 2010) and vision (Jarrett et al., 2009; Ciresan et al., 2011; Rifai et al., 2011a; Krizhevsky et al., 2012).”
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results (2014, background): “…has recently made important progress by adopting, as an accoustic model, deep neural networks such as fully-connected feedforward neural networks (A Mohamed et al. , 2009; Hinton et al. , 2012a), convolutional networks (Sainath et al. , 2014), and recurrent networks (RNN, Graves et al. , 2013a).”
Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding (2016, background): “These are both realised as deep NNs, along with a spiking NN as an incremental and robust encoder of syllable boundaries for coding of continuous fundamental frequency (F0).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “Recently, the deep learning technique was successfully applied to phone recognition [23, 24, 116, 126, 165, 166] and large vocabulary speech recognition tasks [22, 25, 135, 155, 156, 167–169] by integrating the powerful discriminative training ability of DNNs with the sequential modeling ability of ”
Feature engineering in Context-Dependent Deep Neural Networks for conversational speech transcription (2011, background): “RBMs are not scale-invariant, so the training corpus is normalized to zero mean and unit variance [19].”
Hierarchical Convolutional Deep Learning in Computer Vision (2013, background): “[88] proposed to use a system composed of many layers of logistic units.”
A Practical Guide to Training Restricted Boltzmann Machines (2012, background): “…different types of data including labeled or unlabeled images (Hinton et al., 2006a), windows of mel-cepstral coefficients that represent speech (Mohamed et al., 2009), bags of words that represent documents (Salakhutdinov and Hinton, 2009), and user ratings of movies (Salakhutdinov et al.,…”
Analyzing Drum Patterns Using Conditional Deep Belief Networks (2012, background): “Deep belief networks (DBNs) have shown promise in many discriminative tasks, such as written digit recognition [6] and speech recognition [8].”
Calibration of Confidence Measures in Speech Recognition (2011, methodology): “We evaluate the proposed techniques on several datasets in Section VI and conclude the paper in Section VII.”
Deep Neural Networks for Acoustic Modeling in Speech Recognition (2012, methodology): “[12] showed that a DBN-DNN acoustic model outperformed the best published recognition results on TIMIT at about the same time as Sainath et.”
A Bottom-Up Modular Search Approach to Large Vocabulary Continuous Speech Recognition (2013, background): “In addition, detectors based on mechanisms other than feed-forward multi-layer perceptrons, such as deep learning [17], [65], [66], are of great interest as well.”
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition (2012, methodology): “For example, context-independent pre-trained, deep neural network HMM hybrid architectures have recently been proposed for phone recognition [30]–[32] and have achieved very competitive performance.”
Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network (2015, unknown): “The applications of deep learning to image classification [32], [33], language processing [34] and speech recognition [35] have been actively studied in recent years.”