A Fully Differentiable Beam Search Decoder

By Ronan Collobert, Awni Y. Hannun, Gabriel Synnaeve. 2019

43 citations recorded by the source corpus.

We introduce a new beam search decoder that is fully differentiable, making it possible to optimize at training time through the inference procedure. Our decoder allows us to combine models which operate at different granularities (e.g. acoustic and language models). It can be used when target sequences are not aligned to input sequences by considering all possible alignments between the two. We demonstrate our approach scales by applying it to speech recognition, jointly training acoustic and word-level language models. The system is end-to-end, with gradients flowing through the whole architecture from the word-level transcriptions. Recent research efforts have shown that deep neural networks with attention-based mechanisms are powerful enough to successfully train an acoustic model from the final transcription, while implicitly learning a language model. Instead, we show that it is possible to discriminatively train an acoustic model jointly with an explicit and possibly pre-trained language model.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Speech & Audio

Citations in the corpus

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “Nonetheless, there have been increasingly more end-to-end, competitive results, trained with CTC [7, 8], ASG [9], LF-MMI [10], sequence-to-sequence (Seq2Seq) [11, 12], or transducers training [13, 14], and differentiable decoding [15].”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, methodology): “We use wav2letter as accoustic model (Collobert et al., 2016; 2019) and train for 1,000 epochs on 8 GPUs for both TIMIT and WSJ using the auto segmentation criterion.”
wav2vec: Unsupervised Pre-training for Speech Recognition (2019, positive): “Our baseline for the WSJ benchmark is the wav2letter++ setup described in Collobert et al. (2019) which is a 17 layer model with gated convolutions (Dauphin et al., 2017).”
Semi-Supervised Speech Recognition via Local Prior Matching (2020, background): “Luckily, the denominator in equation 2 can often be approximated by summing over a set of hypotheses proposed by a trained ASR model, as done in sequence discriminative training (Povey, 2005; Veselỳ et al., 2013) and differentiable beam search decoding (Collobert et al., 2019).”
Speech Recognition (2018, unknown): “Our baseline for the WSJ benchmark is the wav2letter++ setup described by Collobert et al. (2019) which is a 17 layer model with gated convolutions (Dauphin et al., 2017).”
Word-level Speech Recognition with a Dynamic Lexicon (2019, background): “On the other hand, end-to-end approaches are often non-differentiable [14, 27], or can be quite complex and difficult to implement [12].”
Word-Level Speech Recognition With a Letter to Word Encoder (2019, methodology): “On the other hand, end-to-end approaches are often non-differentiable (Graves & Jaitly, 2014; Prabhavalkar et al., 2018), or can be quite complex and difﬁcult to implement (Collobert et al., 2019).”
Neural Machine Translation: A Review (2019, methodology): “Alternatively, the NMT model training can be tailored towards the decoding algorithm (Goyal et al., 2018; Wiseman & Rush, 2016; Collobert et al., 2019; Gu et al., 2017b).”
Differentiable Weighted Finite-State Transducers (2020, methodology): “In some cases, such as the fully differentiable beam search of Collobert et al. (2019), achieving the necessary computational efﬁciency with a WFST-based implementation may not yet be tractable.”