Linear Networks Based Speaker Adaptation for Speech Synthesis

By Zhiying Huang, Heng Lu, Ming Lei, Zhijie Yan. 2018

14 citations recorded by the source corpus.

Speaker adaptation methods aim to create fair quality synthesis speech voice font for target speakers while only limited resources available. Recently, as deep neural networks based statistical parametric speech synthesis (SPSS) methods become dominant in SPSS TTS back-end modeling, speaker adaptation under the neural network based SPSS framework has also became an important task. In this paper, linear networks (LN) is inserted in multiple neural network layers and fine-tuned together with output layer for best speaker adaptation performance. When adaptation data is extremely small, the low-rank plus diagonal(LRPD) decomposition for LN is employed to make the adapted voice more stable. Speaker adaptation experiments are conducted under a range of adaptation utterances numbers. Moreover, speaker adaptation from 1) female to female, 2) male to female and 3) female to male are investigated. Objective measurement and subjective tests show that LN with LRPD decomposition performs most stable when adaptation data is extremely limited, and our best speaker adaptation (SA) model with only 200 adaptation utterances achieves comparable quality with speaker dependent (SD) model trained with 1000 utterances, in both naturalness and similarity to target speaker.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Junichi Yamagishi
7. Hieu-Thi Luong
8. Yi Ren

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Hieu-Thi Luong
6. Junichi Yamagishi
7. Sheng Zhao
8. Yi Ren

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “, a lot of works in statistic parametric speech synthesis have studied voice adaptation [77, 386, 443, 65, 122], and the recent voice cloning challenge also attracts a lot of participants [388, 118, 331, 45].”
• NAUTILUS: A Versatile Voice Cloning System (2020, background): “on the data of a single speaker [42] or data pooled from multiple speakers [37], [43].”