RobuTrans: A Robust Transformer-Based Text-to-Speech Model

By Naihan Li, Yanqing Liu, Yu Wu, Shujie Liu, Sheng Zhao. 2020

44 citations recorded by the source corpus.

Recently, neural network based speech synthesis has achieved outstanding results, by which the synthesized audios are of excellent quality and naturalness. However, current neural TTS models suffer from the robustness issue, which results in abnormal audios (bad cases) especially for unusual text (unseen context). To build a neural model which can synthesize both natural and stable audios, in this paper, we make a deep analysis of why the previous neural TTS models are not robust, based on which we propose RobuTrans (Robust Transformer), a robust neural TTS model based on Transformer. Comparing to TransformerTTS, our model first converts input texts to linguistic features, including phonemic features and prosodic features, then feed them to the encoder. In the decoder, the encoder-decoder attention is replaced with a duration-based hard attention mechanism, and the causal self-attention is replaced with a "pseudo non-causal attention" mechanism to model the holistic information of the input. Besides, the position embedding is replaced with a 1-D CNN, since it constrains the maximum length of synthesized audio. With these modifications, our model not only fix the robustness problem, but also achieves on parity MOS (4.36) with TransformerTTS (4.37) and Tacotron2 (4.37) on our general set.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation (2023, neutral): “For speech synthesis tasks, AED methods such as Tacotron (Wang et al., 2017), Tacotron2 (Shen et al., 2018), and Trans-formerTTS (Li et al., 2019), and duration-based methods, including Fastspeech (Ren et al., 2019), Fastspeech2 (Ren et al., 2020), and RobuTrans (Li et al., 2020) are widely used.”
A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration FastSpeech [284], DurIAN [411], EATS [67] Glow-TTS [156], [190, 19, 355, 422, 73, 298]”
DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021 (2021, background): “Phoneme representation is widely used as the input of end-toend TTS systems like recent neural TTS models [13, 15, 6, 27], which can avoid ambiguity depending on the text context.”
Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “Moreover, in order to make the self-attention in Transformer more suitable for TTS models, RobuTrans also uses Pseudo Non-causal Attention (PNCA) to replace the traditional causal self-attention.”
Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis (2020, background): “(Li et al. 2020; Chen et al. 2020) identified that the limitation of using location-sensitive attention in the parallel computational model pose a difficulty for the Transformerbased model to learn the alignment between the linguistic and acoustic features.”
HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement (2022, background): “In the ﬁrst stage, low-resolution intermediate representations (e.g., linguistic features, mel-spectrograms) are predicted from text data (Li et al., 2020; Shen et al., 2018; 2020).”
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model (2023, background): “Neural speech synthesis [9, 10, 11, 12, 13, 14, 15, 16, 17, 18] have showed close to human quality in terms of naturalness and audio fidelity in the past years, with cascaded or end-to-end transformer or LSTM architectures.”