Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languages

By Haitong Zhang, Yue Lin. 2020

35 citations recorded by the source corpus.

Recently, sequence-to-sequence models with attention have been successfully applied in Text-to-speech (TTS). These models can generate near-human speech with a large accurately-transcribed speech corpus. However, preparing such a large data-set is both expensive and laborious. To alleviate the problem of heavy data demand, we propose a novel unsupervised pre-training mechanism in this paper. Specifically, we first use Vector-quantization Variational-Autoencoder (VQ-VAE) to ex-tract the unsupervised linguistic units from large-scale, publicly found, and untranscribed speech. We then pre-train the sequence-to-sequence TTS model by using the pairs. Finally, we fine-tune the model with a small amount of paired data from the target speaker. As a result, both objective and subjective evaluations show that our proposed method can synthesize more intelligible and natural speech with the same amount of paired training data. Besides, we extend our proposed method to the hypothesized low-resource languages and verify the effectiveness of the method using objective evaluation.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Sheng Zhao
7. Jingzhou Yang
8. Mutian He

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “In this way, the quantized discrete tokens and the speech can be regarded as pseudo paired data to pre-train a TTS model, which is then fine-tuned on few truly paired text and speech data [197, 352, 429].”
• Multilingual Byte2Speech Text-To-Speech Models Are Few-shot Spoken Language Learners (2021, background): “…noise (Liu et al., 2020b), or leveraging other corpora like fine-tuning autoencoder with discrete latents trained on unpaired target speeches (Zhang & Lin, 2020; Liu et al., 2020a), or dual learning between recognition and synthesis, along with denoising autoencoding (Ren et al., 2019), and…”
• Low-Resource Expressive Text-To-Speech Using Data Augmentation (2020, background): “Unlike many existing low-resource TTS fine-tuning techniques [7, 10, 13], the target data is here already present in the so called pre-training step, making our fine-tuning step more of a refinement step.”