Gpu-Accelerated Viterbi Exact Lattice Decoder for Batched Online and Offline Speech Recognition
By Hugo Braun, Justin Luitjens, Ryan Leary. 2019
14 citations recorded by the source corpus.
We present an optimized weighted finite-state transducer (WFST) decoder capable of online streaming and offline batch processing of audio using Graphics Processing Units (GPUs). The decoder is efficient in memory utilization, input/output (I/O) bandwidth, and uses a novel Viterbi implementation designed to maximize parallelism. The reduced memory footprint allows the decoder to process significantly larger graphs than previously possible, while optimizing I/O increases the number of simultaneous streams supported. GPU preprocessing of lattice segments enables intermediate lattice results to be returned to the requestor during streaming inference. Collectively, the proposed algorithm yields up to a 240x speedup over single core CPU decoding, and up to 40x faster decoding than the current state-of-the-art GPU decoder, while returning equivalent results. This decoder design enables deployment of production-grade ASR models on a large spectrum of systems, ranging from large data center servers to low-power edge devices.
Subfields: Artificial Intelligence, Speech & Audio, ML Systems & Hardware, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Speech & Audio
In ML Systems & Hardware
In Signal Processing
Citations in the corpus
- The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage (2021, methodology): “Acknowledgments and Disclosure of Funding We thank Hugo Braun at NVIDIA for assistance with using the Kaldi cuda decoder.”