Deep Neural Networks for Acoustic Modeling in Speech Recognition

By Geoffrey E. Hinton, Li Deng, Dong Yu, George E. Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew W. Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N. Sainath and 1 more. 2012

2,547 citations recorded by the source corpus.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Generative Adversarial Nets (2014, background): “So far, the most striking successes in deep learning have involved discriminative models, usually those that map a high-dimensional, rich sensory input to a class label [14, 20].”
Sequence to Sequence Learning with Neural Networks (2014, background): “Deep Neural Networks (DNNs) are extremely powerful machine learning models that achieve excellent performance on difficult problems such as speech recognition [13, 7] and visual object recognition [19, 6, 21, 20].”
On the Number of Linear Regions of Deep Neural Networks (2014, background): “Artiﬁcial neural networks with several hidden layers, called deep neural networks, have become popular due to their unprecedented success in a variety of machine learning tasks (see, e.g., Krizhevsky et al. 2012, Ciresan et al. 2012, Goodfellow et al. 2013, Hinton et al. 2012).”
Speech recognition with deep recurrent neural networks (2013, background): “They have gained attention in recent years with the dramatic improvements in acoustic modelling yielded by deep feedforward networks [3, 4].”
Randaugment: Practical automated data augmentation with a reduced search space (2019, background): “D perception [35], speech recognition [19] or audio recognition [18].”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “Deep neural networks were reintroduced in ASR with HMMs [3], and there are plenty of state-of-the-art models relying on force alignment today [4, 5, 6].”
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation (2019, background): “Especially the rise of deep neural networks (NNs) has accelerated this development immensely [1].”
Order Matters: Sequence to sequence for sets (2015, neutral): “Deep architectures have shown in the last few years that they often yield state-of-the-art performance on several tasks, ranging from image classification (Ioffe & Szegedy, 2015) to speech recognition (Hinton et al., 2012).”
Benchmarking Deep Reinforcement Learning for Continuous Control (2016, background): “Recently, signiﬁcant progress has been made by combining advances in deep learning for learning feature representations (Krizhevsky et al., 2012; Hinton et al., 2012) with reinforcement learning, tracing back to much earlier work of Tesauro (1995) and Bert-sekas & Tsitsiklis (1995).”
Deep Convolutional Networks on Graph-Structured Data (2015, background): “These properties are exploited efﬁciently by ConvNets [8, 7], which are designed to extract local features that are shared across the signal domain.”
Local Aggregation for Unsupervised Learning of Visual Embeddings (2019, background): “Deep convolutional neural networks (DCNNs) have achieved great success on many tasks across a variety of domains, such as vision [41, 64, 26, 25, 8], audition [29, 23, 12, 51], and natural language processing [73, 32, 10, 42].”
Generative adversarial networks (2014, neutral): “s in natural language corpora. So far, the most striking successes in deep learning have involved discriminative models, usually those that map a high-dimensional, rich sensory input to a class label [14, 22]. These striking successes have primarily been based on the backpropagation and dropout algo”
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, methodology): “Deep Learning has been applied successfully to Automatic Speech Recognition (ASR) [1], where the main focus of research has been designing better network architectures, for example, DNNs [2], CNNs [3], RNNs [4] and end-to-end models [5, 6, 7].”
Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, methodology): “More recently DNNs have become a fixture in the ASR pipeline with almost all state of the art speech work containing some form of deep neural network (Mohamed et al., 2011; Hinton et al., 2012; Dahl et al., 2011; N. Jaitly & Vanhoucke, 2012; Seide et al., 2011).”
Learning Robust and Multilingual Speech Representations (2020, background): “Unsupervised learning played an import role in the reintroduction of deep networks to speech processing (Hinton et al., 2012), as well as other application areas (Hinton et al.”
Transformers with convolutional context for ASR (2019, background): “Speech Recognition systems have experienced many advances over the past decade, with neural acoustic and language models leading to impressive new levels of performance across many challenging tasks [1, 2].”
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations (2016, background): “ly pushed Articial Intelligence (AI) limits in a wide range of tasks, including but not limited to object recognition from images (Krizhevsky et al., 2012; Szegedy et al., 2014), speech recognition (Hinton et al., 2012; Sainath et al., 2013), statistical machine translation (Devlin et al., 2014; Sut”
Representation Learning: A Review and New Perspectives (2012, background): “…speech recognition, with breakthrough results (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011a; Mohamed et al., 2012; Dahl et al., 2012; Hinton et al., 2012) obtained by several academics as well as researchers at industrial labs bringing these algorithms to a larger scale and into…”