Deep Convolutional Neural Networks with Layer-Wise Context Expansion and Attention

By Dong Yu, Wayne Xiong, Jasha Droppo, Andreas Stolcke, Guoli Ye, Jinyu Li, Geoffry Zweig. 2016

80 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Recent advances in convolutional neural networks (2015, methodology): “ 295]. Besides, small lters have successfully applied in acoustic modeling in hybrid NN-HMM ASR system, and pooling operations are replaced by densely connected layers for ASR tasks [296]. Yu et al . [297] propose a layer-wise context expansion with attention model for ASR. It is a variant of time-d”
Achieving Human Parity in Conversational Speech Recognition (2016, methodology): “The last CNN variant is the LACE (layer-wise context expansion with attention) model [46].”
The microsoft 2016 conversational speech recognition system (2016, methodology): “The last CNN variant is the LACE (layer-wise context expansion with attention) model [17].”
Deep Speaker: an End-to-End Neural Speaker Embedding System (2017, background): “CNNs have also recently been applied to speech recognition with good results [12][13][14][15].”
The Microsoft 2017 Conversational Speech Recognition System (2017, methodology): “LACE, first proposed in [32] and depicted in Figure 1, is a variant of time-delay neural network (TDNN) [4] in which each higher layer is a weighted sum of nonlinear transformations of a window of lower layer frames.”
Comparing Human and Machine Errors in Conversational Speech Transcription (2017, methodology): “Using this methodology, and incorporating state-of-the-art convolutional and recurrent network architectures for both acoustic modeling [10, 11, 8, 12, 13, 14, 15] and language modeling [16, 17, 18, 19] with extensive use of model combination, we obtained a machine error rate that was very slightly ”
A Survey of Deep Convolutional Neural Networks Applied for Prediction of Plant Leaf Diseases (2021, methodology): “Strategies such as skip connections [32], optimization methods [70], transfer learning [69], initialization strategies [71], layer-wise training [72], and Batch Normalization [19] effectively deal with problems such as vanishing gradients, degradation, and internal covariate shift for deep neural ne”
Recent progresses in deep learning based acoustic models (2017, methodology): “Even better, if the deep CNN is designed so that the stride at each layer is long enough to cover the whole kernel, similar to the CNNs with layer-wise context expansion and attention (LACE) [17].”
Single-Channel Multi-talker Speech Recognition with Permutation Invariant Training (2017, background): “Thanks to the signiﬁcant progresses made in the recent years [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], the ASR systems now surpassed the threshold for adoption in many real-world scenarios and enabled services such as Microsoft…”
Toward Human Parity in Conversational Speech Recognition (2017, methodology): “The last CNN variant is the LACE (layer-wise context expansion with attention) model [54], depicted in Fig.”
Residual Convolutional CTC Networks for Automatic Speech Recognition (2017, methodology): “VGG is implemented according to (Yu et al., 2016), which has 14 layers.”