Efficient Implementation of Recurrent Neural Network Transducer in Tensorflow

By Tom Bagby, Kanishka Rao, Khe Chai Sim. 2018

34 citations recorded by the source corpus.

Recurrent neural network transducer (RNN-T) has been successfully applied to automatic speech recognition to jointly learn the acoustic and language model components. The RNN-T loss and its gradient with respect to the softmax outputs can be computed efficiently using a forward-backward algorithm. In this paper, we present an efficient implementation of the RNN-T forward-backward and Viterbi algorithms using standard matrix operations. This allows us to easily implement the algorithm in TensorFlow by making use of the existing hardware-accelerated implementations of these operations. This work is based on a similar technique used in our previous work for computing the connectionist temporal classification and lattice-free maximum mutual information losses, where the forward and backward recursions are viewed as a bi-directional RNN whose states represent the forward and backward probabilities. Our benchmark results on graphic processing unit (GPU) and tensor processing unit (TPU) show that our implementation can achieve better throughput performance by increasing the batch size to maximize parallel computation. Furthermore, our implementation is about twice as fast on TPU compared to GPU for batch

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Personalization of End-to-End Speech Recognition on Mobile Devices for Named Entities (2019, methodology): “The RNN-T loss and gradients are computed using the efficient implementation as described in [28].”
An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models (2019, methodology): “The RNN-T loss and gradients are computed using the efficient implementation as described in [27].”
Recent Advances in End-to-End Automatic Speech Recognition (2021, background): “In [70], to improve training efficiency, the forward and backward probabilities can be vectorized with a loop skewing transformation, and the recursions can be computed in a single loop instead of two nested loops.”
A Comparison of End-to-End Models for Long-Form Speech Recognition (2019, methodology): “All models are implemented with TensorflowLingvo [22], and the RNN-T model further utilize techniques described in [23, 24] to improve training efficiency.”
Knowledge Distillation from Offline to Streaming RNN Transducer for End-to-End Speech Recognition (2020, methodology): “With these definitions, RNN-T training is achieved by minimizing the RNN-T loss LRNN-T that can be efficiently computed by a forwardbackward algorithm [8, 39].”