Monotonic Recurrent Neural Network Transducer and Decoding Strategies

By Anshuman Tripathi, Han Lu, Hasim Sak, Hagen Soltau. 2019

51 citations recorded by the source corpus.

Recurrent Neural Network Transducer (RNNT) is an end-to-end model which transduces discrete input sequences to output sequences by learning alignments between the sequences. In speech recognition tasks we generally have a strictly monotonic alignment between time frames and label sequence. However, the standard RNNT loss does not enforce this constraint. This can cause some anomalies in alignments such as the model outputting a sequence of labels at a single time frame. There is also no bound on the decoding time steps. To address these problems, we introduce a monotonic version of the RNNT loss. Under the assumption that the output sequence is not longer than the input sequence, this loss can be used with forward-backward algorithm to learn strictly monotonic alignments between the sequences. We present experimental studies showing that speech recognition accuracy for monotonic RNNT is equivalent to standard RNNT. We also explore best-first and breadth-first decoding strategies for both monotonic and standard RNNT models. Our experiments show that breadth-first search is effective in exploring and combining alternative alignments. Additionally, it also allows batching of hypotheses during search label expansion, allowing better resource utilization, and resulting in decoding speedup.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Speech & Audio

Citations in the corpus

Fast and Parallel Decoding for Transducer (2022, methodology): “In addition, we use a different transducer architecture from [7] to learn monotonic alignments between the sequence, which shows more promising results.”
End-to-End Speech Recognition: A Survey (2023, background): “The latter constraint – allowing only a single label (blank or non-blank) at each frame – has also been explored in the context of the monotonic RNN-T model [51].”
Less is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging (2020, methodology): “Decoding: All results are reported after decoding models using the breadth-first search decoding algorithm [15].”
Recent Advances in End-to-End Automatic Speech Recognition (2021, background): “Strict alignment between the input speech sequence and output label sequence is enforced in [77] to generate better alignment for streaming RNN-T.”