End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learning
By Tao Tu, Yuan-Jui Chen, Cheng-chieh Yeh, Hung-yi Lee. 2019
100 citations recorded by the source corpus.
End-to-end text-to-speech (TTS) has shown great success on large quantities of paired text plus speech data. However, laborious data collection remains difficult for at least 95% of the languages over the world, which hinders the development of TTS in different languages. In this paper, we aim to build TTS systems for such low-resource (target) languages where only very limited paired data are available. We show such TTS can be effectively constructed by transferring knowledge from a high-resource (source) language. Since the model trained on source language cannot be directly applied to target language due to input space mismatch, we propose a method to learn a mapping between source and target linguistic symbols. Benefiting from this learned mapping, pronunciation information can be preserved throughout the transferring procedure. Preliminary experiments show that we only need around 15 minutes of paired data to obtain a relatively good TTS system. Furthermore, analytic studies demonstrated that the automatically discovered mapping correlate well with the phonetic expertise.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- A Survey on Neural Speech Synthesis (2021, methodology): “[42] propose to map the embeddings between the phoneme sets from different languages, and LRSpeech [390] discards the pre-trained phoneme embeddings and initializes the phoneme embeddings from scratch for low-resource languages.”
- Generative Pre-Training for Speech with Autoregressive Predictive Coding (2019, background): “Finally, we are interested in exploring the usage of APC in other speech applications such as speech synthesis, where pre-training and transfer learning have already achieved some success [33, 34, 35, 36, 37].”
- Semi-Supervised Speaker Adaptation for End-to-End Speech Synthesis with Pretrained Models (2020, methodology): “To solve this problem, adaptation techniques based on fine-tuning have been studied in end-to-end TTS [10, 11], which need only a dozen minutes of single-speaker recordings.”
- Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languages (2020, background): “A recent work [23] investigated transfer learning from high-resource languages to low-resource languages.”
- Distribution Augmentation for Low-Resource Expressive Text-To-Speech (2022, unknown): “One of the most popular classes of methods for improving TTS model performance in low-data settings is transfer learning (TL) [11, 12].”
- Multilingual Byte2Speech Text-To-Speech Models Are Few-shot Spoken Language Learners (2021, methodology): “Also, some methods leverage the phonology of the target language to reduce data requirements (Cai et al., 2020; Chen et al., 2019; Demirsahin et al., 2018), thus linguistic expertise on the language is essential.”
- Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation (2020, background): “In the future, we aim to explore the usage of the proposed method in cross-lingual settings [32, 33].”
- Hierarchical Transfer Learning for Multilingual, Multi-Speaker, and Style Transfer DNN-Based TTS on Low-Resource Languages (2020, methodology): “Some DNN-based systems have successfully applied deep transfer learning, including TTS [31], image classification [34], [35], machine translation [36], [37], automatic speech recognition [38]–[40], language identification [41], and sentiment classification [42].”
- KazakhTTS: An Open-Source Kazakh Text-to-Speech Synthesis Dataset (2021, background): “vised [15], semi-supervised [16], and cross-lingual transfer learning [17] have been developed.”
- Open-source Multi-speaker Speech Corpora for Building Gujarati, Kannada, Malayalam, Marathi, Tamil and Telugu Speech Synthesis Systems (2020, background): “…the found data (Cooper, 2019; Baljekar, 2018), adaptation of the existing corpora to TTS needs (Zen et al., 2019) and development of novel techniques exploiting multilingual sharing, such as transfer learning (Baljekar et al., 2018; Chen et al., 2019; Nachmani and Wolf, 2019; Prakash et al., 2019).”