Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention

By Hideyuki Tachibana, Katsuya Uenoyama, Shunsuke Aihara. 2017

286 citations recorded by the source corpus.

This paper describes a novel text-to-speech (TTS) technique based on deep convolutional neural networks (CNN), without use of any recurrent units. Recurrent neural networks (RNN) have become a standard technique to model sequential data recently, and this technique has been used in some cutting-edge neural TTS techniques. However, training RNN components often requires a very powerful computer, or a very long time, typically several days or weeks. Recent other studies, on the other hand, have shown that CNN-based sequence synthesis can be much faster than RNN-based techniques, because of high parallelizability. The objective of this paper is to show that an alternative neural TTS based only on CNN alleviate these economic costs of training. In our experiment, the proposed Deep Convolutional TTS was sufficiently trained overnight (15 hours), using an ordinary gaming PC equipped with two GPUs, while the quality of the synthesized speech was almost acceptable.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Junichi Yamagishi
6. Jinyu Li
7. Shinji Watanabe
8. Sheng Zhao

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Junichi Yamagishi
5. Frank K. Soong
6. Jinyu Li
7. Shinji Watanabe
8. Lei He

Citations in the corpus

• SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2021, methodology): “We fine-tune the pre-trained model on the 460-hours LibriTTS clean sets (Zen et al., 2019) with the L 1 loss, L loss, and attention loss (Tachibana et al., 2018).”
• A Survey on Neural Speech Synthesis (2021, methodology): “• For the alignment learning between characters/phonemes and mel-spectrograms, the works can be divided into two aspects: 1) enhancing the robustness of attention mechanism [376, 309, 297, 431, 326, 104, 38], and 2) removing attention and instead predicting duration explicitly to bridge the length m”
• NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality (2022, methodology): “We perform transpose convolution for upsampling at every convolution block at a rate of [8, 8, 2, 2].”
• A Comparative Study on Transformer vs RNN in Speech Applications (2019, methodology): “Additionally, we apply a guided attention loss [24] to accelerate the learning of diagonal attention to only the two heads of two layers from the target side.”
• SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model (2021, methodology): “To alleviate possible issues in the attention module we use Double Decoder Consistency (DDC) [22] with gradual training [23] and guided attention [24].”
• MultiSpeech: Multi-Speaker Text to Speech with Transformer (2020, background): “Monotonic and diagonal alignments in the attention weights be-tween text and speech are critical to ensure the quality of synthesized speech [12, 15, 16, 7, 17, 18].”
• Espnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit (2019, background): “v3: Tacotron 2 trained w/ location sensitive attention [32] and the guided attention loss [35] • Transformer.”
• Representation Mixing for TTS Synthesis (2018, unknown): “We approach the alignment problem by jointly learning to align these two types of information [2], effectively translating the information in the linguistic sequence(s) into audio through learned transformations for effective TTS synthesis [3, 4, 5, 6, 7].”
• End-To-End Speech Synthesis Applied to Brazilian Portuguese (2020, methodology): “03 MOS, while the original authors [Tachibana et al., 2017] had 2.”
• Bunched LPCNet : Vocoder for Low-cost Neural Text-To-Speech Systems (2020, methodology): “The results of Anal-Synth using the extracted features from original speech and LPCNet+DCTTS using the predicted features from the DCTTS are summarized in Tables 2 and 3 respectively1.”
• NAUTILUS: A Versatile Voice Cloning System (2020, methodology): “The layers in sdec-prenet use the highway function in the same way as [27] as follows:”
• Light-TTS: Lightweight Multi-Speaker Multi-Lingual Text-to-Speech (2021, methodology): “First, we introduce guided attention loss [16] for better text-to-speech alignments.”
• On the localness modeling for the self-attention based end-to-end speech synthesis (2020, methodology): “To efficiently learn the alignments between two sequences, DCTTS conducts a guided attention constraint to prompt the attention matrix to be ‘nearly diagonal’, which matches the nature of speech production process (Chorowski, Bahdanau, Serdyuk, Cho, & Bengio, 2015; Shen et al., 2017).”
• Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis (2019, background): “where Lmse represents a mean square error for the predicted log mel-spectrogram,Ll1 represents an L1 loss for the predicted log mel-spectrogram, Leos represent binary cross entropy for the probability of the end of the sequence, Latt represents guided attention loss [32], and λ is a hyperparameter f”
• One TTS Alignment to Rule Them All (2021, result): “This idea has been previously explored by Tachibana et al [18] where they introduce a new loss promoting near-diagonal alignments.”