Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine

By George E. Dahl, Marc'Aurelio Ranzato, Abdel-rahman Mohamed, Geoffrey E. Hinton. 2010

341 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, background): “However, the most impressive recent results have been obtained with purely supervised learning techniques for deep networks, in particular for speech recognition (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011) and object recognition (Krizhevsky et al., 2012).”
Unsupervised Pretraining for Sequence to Sequence Learning (2016, neutral): “ the additional language modeling objectives and residual connections. 3 RELATED WORK Pretraining was once an essential technique for training deep networks (Hinton et al., 2006; Bengio et al., 2007; Dahl et al., 2010; Erhan et al., 2010). However, better optimization techniques and big labeled data”
GSNs : Generative Stochastic Networks (2015, background): “recognition (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011) and object recognition (Krizhevsky et al.”
Representation Learning: A Review and New Perspectives (2012, methodology): “While the GRBM has shown considerable potential as the basis of a highly successful phoneme recognition system (Dahl et al., 2010), it seems that due to difficulties in training the mcRBM, the model has been largely superseded by the mPoT model.”
Sequence Transduction with Recurrent Neural Networks (2012, background): “We have introduced a generic sequence transducer composed of two recurrent neural networks and demonstrated its ability to integrate acoustic and linguistic information during a speech recognition task.”
Deep Learning: Methods and Applications (2014, methodology): “[64], the mcRBM parameters in the full DBN are not finetuned using the discriminative information, which is used for fine tuning the higher layers of RBMs, due to the high computational cost.”
Deep Learning using Linear Support Vector Machines (2013, background): “These include (but not limited to) speech (Mohamed et al., 2009; Dahl et al., 2010) and vision (Jarrett et al., 2009; Ciresan et al., 2011; Rifai et al., 2011a; Krizhevsky et al., 2012).”
Convolutional Neural Networks over Tree Structures for Programming Language Processing (2014, background): “Over the past few years, deep learning has made significant breakthroughs in various fields, such as speech recognition (Dahl, Mohamed, and Hinton 2010), computer vision (Krizhevsky, Sutskever, and Hinton 2012), and natural language processing (Collobert and Weston 2008).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “In the architecture of (Dahl et al., 2010), the mcRBM parameters in the full DBN are not easy to be fine-tuned using the discriminative information as for the regular RBMs in the higher layers.”
Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets (2017, background): “Deep Learning models have yielded outstanding results in several applications, including speech recognition [47, 48], computer vision [49, 50, 51], and natural language processing [52, 53, 54, 55].”
Deep learning for visual understanding: A review (2016, methodology): “A new deep architecture has been presented for phone recognition [170] that combines a Mean-Covariance RBM feature extraction module with a standard DBN.”
Application of Pretrained Deep Neural Networks to Large Vocabulary Speech Recognition (2012, methodology): “State of the art results have been reported on phone recognition on TIMIT for a speaker independent, context independent system using a neural network with 8 layers [5].”
An Overview of Deep-Structured Learning for Information Processing (2011, methodology): “The technique of (Mohamed et al., 2009) was improved in the later work reported in (Mohamed et al., 2010) by using the CRF instead of the HMM to model the sequential speech data and by applying the maximum mutual information (MMI) training technique successfully developed in speech recognition to…”
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives (2012, methodology): “It has been used in an object classiﬁ-cation in natural images (Ranzato and Hinton, 2010) as well as the basis of a highly successful phoneme recognition system (Dahl et al. , 2010) whose performance surpassed the previous state-of-the-art in this domain by a signiﬁcant margin.”
General Stochastic Networks for Classification (2014, background): “Deep models in representation learning, also obtain impressive results in supervised learning problems, such as speech recognition, e.g. [7, 8, 9] and computer vision tasks [10].”
Building Program Vector Representations for Deep Learning (2014, background): “…made signiﬁcant breakthroughs in a variety of ﬁelds, such as natural language processing [6], [7], image processing [8], [9], speech recognition [10], [11], etc. Compared with traditional machine learning approaches, deep learning has the following major advantages: • The deep architecture can…”