Deep convolutional neural networks for LVCSR

By Tara N. Sainath, Abdel-rahman Mohamed, Brian Kingsbury, Bhuvana Ramabhadran. 2013

1,136 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dong Yu
2. Jinyu Li
3. Yoshua Bengio
4. Li Deng
5. Quoc V. Le
6. Tara N. Sainath
7. Yu Zhang
8. Jianfeng Gao

In Natural Language Processing

1. Dong Yu
2. Yoshua Bengio
3. Li Deng
4. Quoc V. Le
5. Kyunghyun Cho
6. Jianfeng Gao
7. Mohit Bansal
8. Jinyu Li

In Speech & Audio

1. Jinyu Li
2. Dong Yu
3. Yu Zhang
4. Tara N. Sainath
5. Li Deng
6. Shinji Watanabe
7. Ronan Collobert
8. Yoshua Bengio

In Signal Processing

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Tara N. Sainath
5. Yu Zhang
6. Yoshua Bengio
7. Shinji Watanabe
8. Ronan Collobert

Citations in the corpus

• Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, methodology): “Our neural network architecture was also inspired by past work in convolutions (Sainath et al., 2013) and Transformers (Dong et al.”
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, methodology): “Deep Learning has been applied successfully to Automatic Speech Recognition (ASR) [1], where the main focus of research has been designing better network architectures, for example, DNNs [2], CNNs [3], RNNs [4] and end-to-end models [5, 6, 7].”
• Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, background): “R pipeline with almost all state of the art speech work containing some form of deep neural network [42, 29, 17, 16, 43, 58]. Convolutional networks have also been found beneficial for acoustic models [1, 53]. Recurrent neural networks, typically LSTMs, are just beginning to be deployed in state-of-t”
• Conformer: Convolution-augmented Transformer for Speech Recognition (2020, background): “Recurrent neural networks (RNNs) have been the defacto choice for ASR [1, 2, 3, 4] as they can model the temporal dependencies in the audio sequences effectively [5].”
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations (2016, background): “…including but not limited to object recognition from images (Krizhevsky et al., 2012; Szegedy et al., 2014), speech recognition (Hinton et al., 2012; Sainath et al., 2013), statistical machine translation (Devlin et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015), Atari and Go games…”
• Listen, attend and spell: A neural network for large vocabulary conversational speech recognition (2015, methodology): “They are commonly used in hybrid DNN-HMM speech recognition systems for acoustic modeling [1, 2, 3, 4, 5, 6].”
• Deep Speech: Scaling up end-to-end speech recognition (2014, background): “Speaker adaptation is critical to the success of current ASR systems [44, 36], particularly when trained on 300 hour Switchboard.”
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations (2015, background): “Deep Neural Networks (DNN) have substantially pushed the state-of-the-art in a wide range of tasks, especially in speech recognition [1, 2] and computer vision, notably object recognition from images [3, 4].”
• Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks (2015, neutral): “As a result, DNNs are used in many settings [1], [2], [3], some of which are increasingly security-sensitive [4], [5], [6].”
• Recent advances in convolutional neural networks (2015, methodology): “one convolutional layer in their network, and Amodei et al . [316] use three convolutional layers as the feature preprocessing layer. Recently, very deep CNNs have shown impressive performance in ASR [309, 317{319]. Besides, small lters have successfully applied in acoustic modeling in hybrid NN-HMM”
• Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor (2020, background): “CONVOLUTIONAL neural networks (CNNs) [1], [2], as the most dominant deep learning approaches [3], have been demonstrating their promising performance in addressing the various real-world applications, such as image classification [4], speech understanding [5], and natural language processing [6], to”
• Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition (2017, background): “For example, convolutional neural networks (CNN) which has been huge success in image classification and detection were effective to reduce environmental and speaker variability in acoustic features [6, 7, 8, 9, 10].”
• Achieving Human Parity in Conversational Speech Recognition (2016, methodology): “Compared to earlier applications of CNNs to speech recognition [67, 68], our networks are much deeper, and use linear bypass connections across convolutional layers.”
• Very deep convolutional networks for end-to-end speech recognition (2016, methodology): “Convolutional Neural Networks (CNNs) [8] have been successfully applied to many ASR tasks [9, 10, 11].”
• The microsoft 2016 conversational speech recognition system (2016, methodology): “Compared to earlier applications of CNNs to speech recognition [38, 39], our networks are much deeper, and use linear bypass connections across convolutional layers.”
• Neural Trojans (2017, background): “Highly accurate models can be learned from training samples by neural networks and they have found applications in computer vision, speech recognition, malware detection, etc. [1]–[5].”