Investigation of Enhanced Tacotron Text-to-speech Synthesis Systems with Self-attention for Pitch Accent Language

By Yusuke Yasuda, Xin Eric Wang, Shinji Takaki, Junichi Yamagishi. 2018

89 citations recorded by the source corpus.

End-to-end speech synthesis is a promising approach that directly converts raw text to speech. Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown. Japanese could be one of the most difficult languages for which to achieve end-to-end speech synthesis, largely due to its character diversity and pitch accents. Therefore, state-of-the-art systems are still based on a traditional pipeline framework that requires a separate text analyzer and duration model. Towards end-to-end Japanese speech synthesis, we extend Tacotron to systems with self-attention to capture long-term dependencies related to pitch accents and compare their audio quality with classical pipeline systems under various conditions to show their pros and cons. In a large-scale listening test, we investigated the impacts of the presence of accentual-type labels, the use of force or predicted alignments, and acoustic features used as local condition parameters of the Wavenet vocoder. Our results reveal that although the proposed systems still do not match the quality of a top-line pipeline system for Japanese, we show important stepping stones towards end-to-end Japanese speech synthesis.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram (2019, neutral): “, Japanese) [30].”
Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings (2019, methodology): “For this purpose, we extend an improved Tacotron system in [28] to a multi-speaker TTS system and conduct systematic analysis to answer the above question.”
Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis (2019, methodology): “We chose WaveNet as the benchmark because of its excellent performance reported in both the original paper [12] and our previous study [13][55].”
Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems (2019, methodology): “To improve PAM for VF, it might be better to separately train them with different networks, as in [9, 27, 54].”
Expressive TTS Training With Frame and Style Reconstruction Loss (2020, background): “To improve the suprasegmental expressiveness, there have been studies [32], [35], [51] on latent prosody representations, that make possible prosody styling in Tacotron TTS framework.”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “The above mentioned powerful end-to-end speech synthesizer models [120, 124] have enabled the production of large scale commercial products, such as Google Cloud TTS, Amazon AWS Polly, and Baidu TTS.”
A Review of Deep Learning Based Speech Synthesis (2019, background): “In addition, the authors in [73] also proposed the Tacotron2 system to generate audio signals that resulted in a very high mean opinion score (MOS) comparable to human speech [74].”
Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS (2020, methodology): “Others [36], [37] just take the prosody embeddings as an auxiliary input to the TTS model.”
Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis (2019, methodology): “[23] used additional accent information to extract an accent embedding, which forms an additional input to the encoder.”
The STC text-to-speech system for Blizzard Challenge 2019 (2019, unknown): “First, it has been proven to be suitable as a base for making TTS systems for different languages (e.g. English [3], Japanese [23] and even multi-language ones (English, Spanish and Chinese in [24]).”
Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments (2019, methodology): “The scores of the baseline systems are consistent with the previous work [17].”