SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

By Edresson Casanova, Christopher Dane Shulby, Eren Gölge, N. Müller, Frederico Santos de Oliveira, Arnaldo Cândido Júnior, A. S. Soares, Sandra Maria Aluísio, Moacir Antonelli Ponti. 2021

128 citations recorded by the source corpus.

In this paper, we propose SC-GlowTTS: an efficient zero-shot multi-speaker text-to-speech model that improves similarity for speakers unseen during training. We propose a speaker-conditional architecture that explores a flow-based decoder that works in a zero-shot scenario. As text encoders, we explore a dilated residual convolutional-based encoder, gated convolutional-based encoder, and transformer-based encoder. Additionally, we have shown that adjusting a GAN-based vocoder for the spectrograms predicted by the TTS model on the training dataset can significantly improve the similarity and speech quality for new speakers. Our model converges using only 11 speakers, reaching state-of-the-art results for similarity with new speakers, as well as high speech quality.

Subfields: Artificial Intelligence, Natural Language Generation, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Zhou Zhao
5. Yi Ren
6. Rongjie Huang
7. Jinglin Liu
8. Frank K. Soong

In Natural Language Generation

1. Dong Yu
2. Helen Mei-Ling Meng
3. Frank K. Soong
4. Songxiang Liu

In Speech & Audio

1. Xu Tan
2. Yi Ren
3. Zhou Zhao
4. Tao Qin
5. Rongjie Huang
6. Tie-Yan Liu
7. Jinglin Liu
8. Dongchao Yang

In Signal Processing

1. Rongjie Huang
2. Yi Ren
3. Xu Tan
4. Zhou Zhao
5. Jinglin Liu
6. Tie-Yan Liu
7. Tao Qin
8. Frank K. Soong

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “Some works [9, 43, 139, 55, 32] conduct zero-shot adaptation, which leverage a speaker encoder to extract speaker embeddings given a reference audio.”
• InstructTTS: Modelling Expressive TTS in Discrete Latent Space With Natural Language Style Prompt (2023, background): “In the field of expressive speech synthesis, several works [6], [9], [12], [27], [28], [29] have been done.”
• GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech Synthesis (2022, background): “SC-GlowTTS [3] proposes a speaker-conditional architecture that explores a flow-based decoder in a zero-shot scenario.”
• XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model (2024, methodology): “This approach is called zero-shot multi-speaker TTS (ZS-TTS) as in [1, 2, 3, 4, 5, 6].”
• Nonparallel Expressive TTS for Unseen Target Speaker using Style-Controlled Adaptive Layer and Optimized Pitch Embedding (2023, methodology): “SC-GlowTTS [19] proposed a speaker-conditional architecture using a flow-based decoder in a zero-shot scenario.”