First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs

By Andrew L. Maas, Awni Y. Hannun, Dan Jurafsky, Andrew Y. Ng. 2014

173 citations recorded by the source corpus.

We present a method to perform first-pass large vocabulary co ntinuous speech recognition using only a neural network and language model. Deep neural network acoustic models are now commonplace in HMM-based speech recognition systems, but building such systems is a complex, domain-specific task. Recent work demonstrated the feasibility of discarding the HMM sequence modeling framework by directly predicting transcript text from audio. This paper extends this approach in two ways. First, we demonstrate that a straightforward recurrent neural network architecture can achieve a high level of accuracy. Second, we propose and evaluate a modified prefix-search decoding algorith m. This approach to decoding enables first-pass speech recognition with a langu age model, completely unaided by the cumbersome infrastructure of HMM-based systems. Experiments on the Wall Street Journal corpus demonstrate fairly competitive word error rates, and the importance of bi-directional network recurrence.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, methodology): “We use a beam search to find the optimal transcription [27].”
Deep Speech: Scaling up end-to-end speech recognition (2014, methodology): “Our recurrent network is similar to the bidirectional RNN used by Hannun et al. [16], but with multiple changes to enhance its scalability.”
End-to-end attention-based large vocabulary speech recognition (2015, methodology): “In this work, we investigate the application of an Attentionbased Recurrent Sequence Generator (ARSG) as a part of an end-to-end LVCSR system.”
cuDNN: Efficient Primitives for Deep Learning (2014, background): “Deep neural networks for speech recognition have also benefited from parallel implementations on GPUs [10] [9] [15].”
EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding (2015, methodology): “Our experiments with the Wall Street Journal (WSJ) benchmark show that Eesen results in superior performance than the existing end-to-end ASR pipelines [6, 8].”
Letter-Based Speech Recognition with Gated ConvNets (2017, background): “Competitive end-to-end approaches leverage acoustic models (often ConvNet-based) topped with RNN layers as in (Hannun et al., 2014a; Miao et al., 2015; Saon et al., 2015; Amodei et al., 2016; Zhou et al., 2018; Zeyer et al., 2018) (e.g. a state of the art on WSJ (Chan and Lane, 2015a)), trained…”
Exploring neural transducers for end-to-end speech recognition (2017, methodology): “This decoding is approximate, and performed using beam search, typically with a large beam or lattice [14, 18].”
Attention-based Wav2Text with feature transfer learning (2017, methodology): “We used the character sequence as our decoder target and followed the preprocessing step proposed by [20].”
Maximum a posteriori Based Decoding for CTC Acoustic Models (2016, methodology): “Recently, the connectionist temporal classification (CTC) [16], which enables a neural network to learn the direct mapping from audio input X to symbol sequence s, has been successfully applied to large vocabulary continuous speech recognition (LVCSR) tasks [17, 18, 19, 20, 21, 22, 23].”
Sequence-to-Sequence Asr Optimization Via Reinforcement Learning (2017, methodology): “We used the character sequence as our decoder target and followed the preprocessing steps proposed by [17].”
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition (2019, methodology): “ units (GRUs) [8,30]; furthermore, the success of the ReLU(x) = max(0;x)nonlinearity in preventing vanishing gradients enabled the use of vanilla bidirectional recurrent deep neural networks (BRDNNs) [5,6,31] to reduce operations per layer. Convolutions over time and frequency were ﬁrst used as init”
Neural Phrase-based Machine Translation (2017, background): “As reported in Maas et al. (2014); Bahdanau et al. (2017), we find that penalizing candidate sentences that are too short was required to obtain the best results.”
Towards Neural Phrase-based Machine Translation (2017, background): “As reported in Maas et al. (2014); Bahdanau et al. (2017), we find that penalizing candidate sentences that are too short was required to obtain the best results.”