Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks

By Tara N. Sainath, Oriol Vinyals, Andrew W. Senior, Hasim Sak. 2015

1,534 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Model Architectures, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Signal Processing

Citations in the corpus

Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, methodology): “Additionally, we dynamically augment the dataset by adding unique noise every epoch with an SNR between 0dB and 30dB, just as in (Hannun et al., 2014a; Sainath et al., 2015).”
Listen, attend and spell: A neural network for large vocabulary conversational speech recognition (2015, methodology): “This increased the amount of audio data by 20 times with a SNR between 5dB and 30dB [22].”
The Limitations of Deep Learning in Adversarial Settings (2015, methodology): “accuracy rates higher than previous classiﬁcation techniques. In short, DL is transforming computational processing of complex data in many domains such as vision [24], [37], speech recognition [15], [32], [33], language processing [13], ﬁnancial fraud detection [23], and recently malware detection ”
Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition (2017, background): “For example, convolutional neural networks (CNN) which has been huge success in image classification and detection were effective to reduce environmental and speaker variability in acoustic features [6, 7, 8, 9, 10].”
A New Representation of Skeleton Sequences for 3D Action Recognition (2017, background): “, it is still difﬁcult for LSTM to memorize the information of the entire sequence with many timesteps [48, 13]. In addition, it is also difﬁcult to construct deep LSTM to extract high-level features [35, 30]. Convolutional neural networks (CNNs) [24] nowadays have achieved great success in image cl”
FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, background): “This is interesting as CRNNs were originally proposed for speech recognition [124] before being adapted for SER [20].”
EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding (2015, methodology): “We exploit deep recurrent neural networks (RNNs) [15, ar X iv :1 50 7.”
CNN architectures for large-scale audio classification (2016, methodology): “RNNs and CNNs have also been applied to Large Vocabulary Continuous Speech Recognition (LVCSR) [18].”
Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform (2015, background): “h long short-term memory (LSTM) units [20] or gated recurrent units (GRUs) [8, 9] have proven successful to model the long term dependencies in sequential data (e.g., text and speech). Sainath et al. [37] have combined CNNs and RNNs into one uniﬁed architecture for speech recognition. Some recent wo”
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers (2016, background): “For example, (Chen et al., 2015; Visin et al., 2015) combined convolution and recurrent layers to tackle image segmentation tasks; (Sainath et al., 2015) applied a similar model to do speech recognition.”
Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems (2017, positive): “The first convolution further improves the results, in line with previous findings about convolutions as feature extractors before recurrent layers [36].”
A C-LSTM Neural Network for Text Classification (2015, background): “(Sainath et al., 2015) have obtained respectable improvements in WER by learning a deep LSTM from multi-scale inputs.”
A Comparison of Sequence-to-Sequence Models for Speech Recognition (2017, background): “Most state-of-the-art automatic speech recognition (ASR) systems are comprised of separate acoustic, pronunciation, and language modeling components that are trained independently [1, 2].”
Deep-Convolution-Based LSTM Network for Remaining Useful Life Prediction (2021, background): “Like traditional LSTM, CLSTM can be employed as a building block for deep architecture, which has been proven to be highly effective in various application due to its ability to learn hierarchical representation of the raw data [32].”
On the compression of recurrent neural networks with an application to LVCSR acoustic modeling for embedded speech recognition (2016, methodology): “Index Terms— model compression, LSTM, RNN, SVD, embedded speech recognition”
From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition (2019, methodology): “DNN is subsequently replaced with variants of Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) [5–10], whose primary advantage comes from their ability to model long temporal context.”