Residual Convolutional CTC Networks for Automatic Speech Recognition

By Yisen Wang, Xuejiao Deng, Songbai Pu, Zhiheng Huang. 2017

90 citations recorded by the source corpus.

Deep learning approaches have been widely used in Automatic Speech Recognition (ASR) and they have achieved a significant accuracy improvement. Especially, Convolutional Neural Networks (CNNs) have been revisited in ASR recently. However, most CNNs used in existing work have less than 10 layers which may not be deep enough to capture all human speech signal information. In this paper, we propose a novel deep and wide CNN architecture denoted as RCNN-CTC, which has residual connections and Connectionist Temporal Classification (CTC) loss function. RCNN-CTC is an end-to-end system which can exploit temporal and spectral structures of speech signals simultaneously. Furthermore, we introduce a CTC-based system combination, which is different from the conventional frame-wise senone-based one. The basic subsystems adopted in the combination are different types and thus mutually complementary to each other. Experimental results show that our proposed single system RCNN-CTC can achieve the lowest word error rate (WER) on WSJ and Tencent Chat data sets, compared to several widely used neural network systems in ASR. In addition, the proposed system combination can offer a further error reduction on these two data sets, resulting in relative WER reductions of $14.91\%$ and $6.52\%$ on WSJ dev93 and Tencent Chat data sets respectively.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Speech & Audio

In Signal Processing

Citations in the corpus

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “, 2016) and have since been successfully applied to speech recognition (Xiong et al., 2017; Saon et al., 2017; Li et al., 2019b; Wang et al., 2017).”
ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism (2017, methodology): “Besides, convolutional architectures initially developed for image classiﬁcation such as Residual Networks are now routinely used for machine translation [13] and speech recognition [45].”
Deep Speaker: an End-to-End Neural Speaker Embedding System (2017, background): “CNNs have also recently been applied to speech recognition with good results [12][13][14][15].”
Deep Audio-Visual Speech Recognition (2018, methodology): “For example, fully convolutional networks have been used for ASR with CTC [51], [55] or a simplified vari-”
ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases (2017, methodology): “Besides, convolutional architectures initially developed for image classification such as Residual Networks are now routinely used for machine translation [11] and speech recognition [12].”
Adversarial Weight Perturbation Helps Robust Generalization (2020, background): “Although deep neural networks (DNNs) have been widely deployed in a number of fields such as computer vision [14], speech recognition [51], and natural language processing [10], they could be easily fooled to confidently make incorrect predictions by adversarial examples that are crafted by adding i”
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition (2019, methodology): “…end-to-end approach to monotonic sequence transduction tasks like handwriting recognition [2], action labeling [3], and automatic speech recognition (ASR) [1, 4–11], sidestepping the label alignment procedure required by traditional hidden Markov model plus neural network (HMM-NN) approaches [12].”
Adversarial Neuron Pruning Purifies Backdoored Deep Models (2021, background): “In recent years, deep neural networks (DNNs) achieve satisfactory performance in many tasks, including computer vision [15], speech recognition [45], and gaming agents [36].”
Improving Adversarial Robustness Requires Revisiting Misclassified Examples (2020, background): “Despite their great success in applications such as computer vision (He et al., 2016), speech recognition (Wang et al., 2017) and natural language processing (Devlin et al., 2018; Zeng et al., 2019), deep neural networks (DNNs) are extremely vulnerable to adversarial examples crafted by adding…”