An Analysis of "Attention" in Sequence-to-Sequence Models

45 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese (2018, background): “A S] 1 8 M ay 2 01 8 thermore, the modeling units of WPM have been explored in [6], which are sub-word units ranging from graphemes all the way up to entire words.”
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese (2018, background): “An attention layer acts as the interface between the encoder and the decoder: it selects frames in the encoder representation that the decoder should attend to in order to predict the next sub-word unit [5].”
Streaming small-footprint keyword spotting using sequence-to-sequence models (2017, methodology): “We have also shown, that sequence-to-sequence models trained to predict phoneme-based targets, can be effective when used in a second pass rescoring framework [28].”
Two-Pass End-to-End Speech Recognition (2019, methodology): “Language model rescoring is commonly been used for multi-pass decoding [14, 15, 16, 17], but more recently has been used with a LAS model to rescore hypotheses from a first-pass conventional model [18].”