An Online Sequence-to-Sequence Model Using Partial Conditioning

By Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, Ilya Sutskever, David Sussillo, Samy Bengio. 2015

114 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Efficient Content-Based Sparse Attention with Routing Transformers (2020, background): “In the context of speech recognition, Jaitly et al. (2016) proposed the Neural Transducer which segments sequences in non-overlapping chunks and attention is performed in each chunk independently.”
State-of-the-Art Speech Recognition with Sequence-to-Sequence Models (2017, background): “Therefore, an important next step is to revise this model with an streaming attention-based model, such as Neural Transducer [33].”
Exploring architectures, data and units for streaming end-to-end speech recognition with RNN-transducer (2017, background): “Several streaming encoder-decoder architectures have been proposed previously, including the neural transducer [11], the recurrent neural aligner (RNA) [12], and the”
Improved training of end-to-end attention models for speech recognition (2018, positive): “We use the MLP attention [20,21,31,32,51].”
Extending Recurrent Neural Aligner for Streaming End-to-End Speech Recognition in Mandarin (2018, background): “Recently, a considerable amount of works have demonstrated the simpliﬁcation and effectiveness of end-to-end models in the ASR ﬁeld [1, 2, 3, 4, 5].”
Improving Unsupervised Style Transfer in end-to-end Speech Synthesis with end-to-end Speech Recognition (2018, methodology): “The TTS model we used here is GST-tacotron [8], which is already introduced in Section 3, while ASR model can be any end-toend seq2seq model[28, 29, 30, 31, 32, 33, 34].”
A Better and Faster end-to-end Model for Streaming ASR (2020, methodology): “Models such as Recurrent Neural Network Transducer (RNN-T) [9], Neural Transducer [12, 13], Monotonic Attention [14, 15], Triggered attention [16] and Scout Network [17] have been explored to address the streaming nature.”
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (2018, background): “Jaitly et al. (2016) propose an online neural transducer for speech recognition that is conditioned on prefixes.”
A Comparison of Sequence-to-Sequence Models for Speech Recognition (2017, background): “We note, however, that recent work on “online” attention [16] is a step towards enabling streaming attention-based models.”
Improving Attention-Based End-to-End ASR Systems with Sequence-Based Loss Functions (2018, background): “To enforce the temporal constraints inherent in speech recognition tasks, various attention mechanisms are proposed [19, 20].”
Twin Regularization for online speech recognition (2018, methodology): “Attempts have also been made to develop robust online speech recognizers based on RNNs, exploiting both the traditional RNN-HMM framework [14–19] and, more recently, end-to-end ASR technology [20, 21].”
An Analysis of "Attention" in Sequence-to-Sequence Models (2017, background): “[4] proposed the Neural Transducer (NT) model, that limits the number of encoder frames that the model can attend to in predicting the next output unit.”
s-Transformer: Segment-Transformer for Robust Neural Speech Synthesis (2020, methodology): “We have tried segment based Tacotron model using partial conditioning with incremental prediction[14], but due to the size limitation of previous RNN state, the experiment reports inferior performance.”
Self-attention Aligner: A Latency-control End-to-end Model for ASR Using Self-attention Network and Chunk-hopping (2019, methodology): “End-to-end models [1, 2, 3, 4, 5] have greatly simplified the automatic speech recognition (ASR) system by combining acoustic model, language model and an acoustic-to-text alignment mechanism (e.”
Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS (2019, methodology): “As for locality, methods like [5-8] are proposed, while monotonicity and completeness could be further improved by explicitly modelling alignments over memories with non-decreasing position [9-12].”
First Step Towards End-to-End Parametric TTS Synthesis: Generating Spectral Parameters with Neural Attention (2016, methodology): “2A similar modulator using LSTM was proposed to compute attention scores in [25].”