Semi-Supervised Speaker Adaptation for End-to-End Speech Synthesis with Pretrained Models

By Katsuki Inoue, Sunao Hara, Masanobu Abe, Tomoki Hayashi, Ryuichi Yamamoto, Shinji Watanabe. 2020

14 citations recorded by the source corpus.

Recently, end-to-end text-to-speech (TTS) models have achieved a remarkable performance, however, requiring a large amount of paired text and speech data for training. On the other hand, we can easily collect unpaired dozen minutes of speech recordings for a target speaker without corresponding text data. To make use of such accessible data, the proposed method leverages the recent great success of state-of-the-art end-to-end automatic speech recognition (ASR) systems and obtains corresponding transcriptions from pretrained ASR models. Although these models could only provide text output instead of intermediate linguistic features like phonemes, end-to-end TTS can be well trained with such raw text data directly. Thus, the proposed method can greatly simplify a speaker adaptation pipeline by consistently employing end-to-end ASR/TTS ecosystems. The experimental results show that our proposed method achieved comparable performance to a paired data adaptation method in terms of subjective speaker similarity and objective cepstral distance measures.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Yi Ren
7. Zhou Zhao
8. Lei He

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Frank K. Soong
5. Sheng Zhao
6. Yi Ren
7. Zhou Zhao
8. Lei He

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Efficient Adaptation Few-Data Adaptation [43, 9, 173, 234, 439, 48, 39, 230] Untranscribed Data Adaptation [397, 130, 215] Few-Parameter Adaptation [9, 43, 39] Zero-Shot Adaptation [9, 43, 139, 55]”
• Cross-Lingual Speaker Adaptation Using Domain Adaptation and Speaker Consistency Loss for Text-To-Speech Synthesis (2021, unknown): “Existing monolingual speaker adaptation methods often use a small amount of the target speaker’s speech data to fine-tune a pretrained multi-speaker TTS model [6, 7].”