New types of deep neural network learning for speech recognition and related applications: an overview

By Li Deng, Geoffrey E. Hinton, Brian Kingsbury. 2013

1,215 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, methodology): “Multitask learning (Caruana, 1997) has been successfully applied to multiple domains, including NLP, speech processing, drug discovery and many others (Collobert and Weston, 2008; Deng et al., 2013; Ramsundar et al., 2015; Maurer et al., 2016; Ruder, 2017; Kaiser et al., 2017).”
SGDR: Stochastic Gradient Descent with Warm Restarts (2016, methodology): “Deep neural networks (DNNs) are currently the best-performing method for many classiﬁcation problems, such as object recognition from images (Krizhevsky et al., 2012a; Donahue et al., 2014) or speech recognition from audio data (Deng et al., 2013).”
Local Aggregation for Unsupervised Learning of Visual Embeddings (2019, background): “Deep convolutional neural networks (DCNNs) have achieved great success on many tasks across a variety of domains, such as vision [41, 64, 26, 25, 8], audition [29, 23, 12, 51], and natural language processing [73, 32, 10, 42].”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “hree dierent research groups on three standard speech recognition benchmarks. For all three groups, deep neural networks (DNNs) signicantly outperform the traditional Gaussian mixture models (GMMs) [50]. These experiments were performed between 2010 and 2012 and were instrumental in the recent reviv”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “Machine learning models, especially deep neural networks, have been deployed prominently in many real-world applications, such as image classification [28, 49], speech recognition [11, 21], natural language processing [2, 10], and game playing [35, 48].”
Fixed Point Quantization of Deep Convolutional Networks (2015, background): “More recently, Lin et al. (2015) showed that deep neural networks can be effectively trained using only binary weights, which in some cases can even improve classiﬁcation accuracy relative to the ﬂoating point baseline.”
Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models (2017, background): “Today’s machine learning-based AI systems excel in a number of complex tasks ranging from the detection of objects in images [14] and the understanding of natural languages [8] to the processing of speech signals [10].”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “ests in machine learning and pattern recognition, for example image recognition (He et al., 2016), object detection (Ren et al., 2015), semantics segmentation (Chen et al., 2016a), audio recognition (Deng et al., 2013), and natural language processing (Cambria and White, 2014). Deep-learning is capa”
Deep Learning: Methods and Applications (2014, methodology): “This success is a result of close academic-industrial collaboration, initiated at Microsoft Research, with the involved researchers identifying and acutely attending to the industrial need for large-scale deployment [68, 89, 109, 161, 323, 414].”
Massively Multitask Networks for Drug Discovery (2015, methodology): “Prior work has used multitask deep networks in the contexts of language understanding (Collobert & Weston, 2008) and multi-language speech recognition (Deng et al., 2013).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “, 2013; Alpaydin, 2014) and neural learning (Deng et al., 2013; LeCun et al., 2015; Silver et al., 2016), highlight the importance of learning from large input-output data and extensive training.”
Convolutional Neural Network for Paraphrase Identification (2015, methodology): “, 2014), speech recognition (Deng et al., 2013) and natural language processing (Collobert and Weston, 2008), we adopt a deep learning approach to paraphrase identification in this paper.”
Modeling Interestingness with Deep Neural Networks (2014, neutral): “These keywords represent the entities or concepts that might interest users, via the convolutional and max-pooling layers which are related to the deep models used for computer vision (Krizhevsky et al., 2013) and speech recognition (Deng et al., 2013a) but are not used in Huang et al.’s model.”
Embedding Entities and Relations for Learning and Inference in Knowledge Bases (2014, background): “As learning representations using deep networks has shown great success in various applications (Hinton et al., 2012; Vinyals et al., 2012; Deng et al., 2013), it may also help capturing hierarchical structure hidden in the multi-relational data.”
DeepTox: Toxicity Prediction using Deep Learning (2016, background): “…fields, redefining the state of the art in vision (Cire¸san et al., 2012a; Krizhevsky et al., 2012), speech recognition (Dahl et al., 2012; Deng et al., 2013; Graves et al., 2013), text understanding and natural language processing (Socher and Manning, 2013; Sutskever et al., 2014),…”
Online Batch Selection for Faster Training of Neural Networks (2015, methodology): “, 2014) or speech recognition from audio data (Deng et al., 2013).”