Forward Attention in Sequence- To-Sequence Acoustic Modeling for Speech Synthesis

By Jing-Xuan Zhang, Zhenhua Ling, Lirong Dai. 2018

86 citations recorded by the source corpus.

This paper proposes a forward attention method for the sequence-to-sequence acoustic modeling of speech synthesis. This method is motivated by the nature of the monotonic alignment from phone sequences to acoustic sequences. Only the alignment paths that satisfy the monotonic condition are taken into consideration at each decoder timestep. The modified attention probabilities at each timestep are computed recursively using a forward algorithm. A transition agent for forward attention is further proposed, which helps the attention mechanism to make decisions whether to move forward or stay at each decoder timestep. Experimental results show that the proposed forward attention method achieves faster convergence speed and higher stability than the baseline attention method. Besides, the method of forward attention with transition agent can also help improve the naturalness of synthetic speech and control the speed of synthetic speech effectively.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Survey on Neural Speech Synthesis (2021, background): “• For the alignment learning between characters/phonemes and mel-spectrograms, the works can be divided into two aspects: 1) enhancing the robustness of attention mechanism [376, 309, 297, 431, 326, 104, 38], and 2) removing attention and instead predicting duration explicitly to bridge the length m”
End-to-End Adversarial Text-to-Speech (2020, background): “Several mechanisms have been proposed to exploit monotonicity in tasks that require sequence alignment, including attention mechanisms (Graves, 2013; Zhang et al., 2018; Vasquez & Lewis, 2019; He et al., 2019; Raffel et al., 2017; Chiu & Raffel, 2018), loss functions (Graves et al.”
Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings (2019, background): “The LSTM output is passed to a forward attention block [33], which speeds up the alignment, and the self-attention output is passed to an additive attention block, which allows attention to longer-range information.”
Location-Relative Attention Mechanisms for Robust Long-Form Speech Synthesis (2019, background): “The hybrid mechanism still has occasional alignment issues which led a number of authors to develop attention mechanisms that directly exploit monotonicity [9, 5, 6].”
MultiSpeech: Multi-Speaker Text to Speech with Transformer (2020, background): “Monotonic and diagonal alignments in the attention weights be-tween text and speech are critical to ensure the quality of synthesized speech [12, 15, 16, 7, 17, 18].”
Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS (2019, methodology): “Particularly, forward attention is proposed to stabilize the attention process in TTS, which reweighs the alignments by previous ones using forward variables, possibly modulated by a “transition agent” [14].”
s-Transformer: Segment-Transformer for Robust Neural Speech Synthesis (2020, background): “Various attention mechanisms are tried in the encoder-decoder attention model, like stepwise monotonic attention [10], forward-Samples are available at https://vancycici.github.io/sTransformer/ attention[11], monotonic chunkwise attention [12], and etc.”
The STC text-to-speech system for Blizzard Challenge 2019 (2019, unknown): “Second, it is flexible: there is a lot of research into various add-ons and modifications to this architecture, aimed at improving overall quality and/or making some kind of prosody control feasible [25, 24, 26], improving stability [27], semi-supervised learning, transfer learning and efficient use”
Triple M: A Practical Text-to-Speech Synthesis System with Multi-Guidance Attention and Multi-Band Multi-Time LPCNet (2021, background): “In order to alleviate these problems, alternative attention mechanisms such as forward attention [5], GMM-based attention [6], stepwise monotonic attention [7] and dynamic convolution attention [8] are proposed.”
VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention (2021, background): “, 2015), forward attention (Zhang et al., 2018), multi-head attention (Li et al.”
Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments (2019, methodology): “Such attempts include location-sensitive attention [8], monotonic attention [9], and forward attention [10].”
Review of end-to-end speech synthesis technology based on deep learning (2021, background): “[259] suggested that only the alignment paths satisfying the monotonic condition are taken into consideration at each decoder time step.”
Using IPA-Based Tacotron for Data Efficient Cross-Lingual Speaker Adaptation and Pronunciation Enhancement (2020, methodology): “We use Forward attention [12] in the attention layer of our model.”
JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment (2020, methodology): “The algorithm has same procedures as in original paper [12] except that it prepares the attention weights w1:T (1 : N) for all time steps N,T in parallel using the attention mechanism Attention.”
Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit (2020, background): “nference, which leads to skipping or repeating words, incomplete synthesis, or an inability to synthesize long utterances [12]. To address the issue, many monotonic attention mechanisms are presented [13, 14], where only the alignment paths satisfying the monotonic condition are taken into considera”
Non-Autoregressive Transformer Automatic Speech Recognition (2019, unknown): “The problem is similar to words skipping problem in text-to-speech [17].”