Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding

By Seungwoo Choi, Seungju Han, Dongyoung Kim, Sungjoo Ha. 2020

76 citations recorded by the source corpus.

On account of growing demands for personalization, the need for a so-called few-shot TTS system that clones speakers with only a few data is emerging. To address this issue, we propose Attentron, a few-shot TTS model that clones voices of speakers unseen during training. It introduces two special encoders, each serving different purposes. A fine-grained encoder extracts variable-length style information via an attention mechanism, and a coarse-grained encoder greatly stabilizes the speech synthesis, circumventing unintelligible gibberish even for synthesizing speech of unseen speakers. In addition, the model can scale out to an arbitrary number of reference audios to improve the quality of the synthesized speech. According to our experiments, including a human evaluation, the proposed model significantly outperforms state-of-the-art models when generating speech for unseen speakers in terms of speaker similarity and quality.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Hung-yi Lee
7. Da-Rong Liu
8. Yi-Chen Chen

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Frank K. Soong
5. Hung-yi Lee
6. Yi-Chen Chen
7. Sung-Feng Huang
8. Da-Rong Liu

Citations in the corpus

• YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, positive): “Following the previous works [4, 10], we compute SECS using the speaker encoder of the Resemblyzer [42] package; thus, allowing comparison with those studies.”
• Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, background): “There are two general approaches to deal with such task [8], speaker adaptation [8], [9], [10], [11], [12] and speaker encoding [6], [13], [7], [14], [15], [16], [17].”
• A Survey on Neural Speech Synthesis (2021, methodology): “Some works [43, 9, 173, 234, 439, 48, 45, 39, 230] conduct few-shot adaptation that only uses few paired text and speech data, varying from several minutes to several seconds.”
• SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model (2021, methodology): “Despite the advantage of the few-shot approach, our model still achieves a higher SECS than Attentron.”
• Effective and Differentiated Use of Control Information for Multi-speaker Speech Synthesis (2021, background): “[10] proposed an attentionbased variable-length embedding.”
• The Multi-Speaker Multi-Style Voice Cloning Challenge 2021 (2021, unknown): “In [17], a fine-grained encoder extracts variable-length style information via attention mechanism and a coarse-grained encoder greatly stabilizes the speech synthesis to improve robustness.”