Unified Mandarin TTS Front-end Based on Distilled BERT Model

By Yang Zhang, Liqun Deng, Yasheng Wang. 2020

26 citations recorded by the source corpus.

The front-end module in a typical Mandarin text-to-speech system (TTS) is composed of a long pipeline of text processing components, which requires extensive efforts to build and is prone to large accumulative model size and cascade errors. In this paper, a pre-trained language model (PLM) based model is proposed to simultaneously tackle the two most important tasks in TTS front-end, i.e., prosodic structure prediction (PSP) and grapheme-to-phoneme (G2P) conversion. We use a pre-trained Chinese BERT[1] as the text encoder and employ multi-task learning technique to adapt it to the two TTS front-end tasks. Then, the BERT encoder is distilled into a smaller model by employing a knowledge distillation technique called TinyBERT[2], making the whole model size 25% of that of benchmark pipeline models while maintaining competitive performance on both tasks. With the proposed the methods, we are able to run the whole TTS front-end module in a light and unified manner, which is more friendly to deployment on mobile devices.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Xinyu Yang
6. Zhaoxi Mu
7. Yizhuo Dong
8. Sheng Zhao

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “TTS Text Analysis Char→Linguistic TN [310, 223, 430], G2P [403, 320, 321] Prosody Prediction [312, 137, 277, 251] Unified Model [252, 437] DeepVoice 1/2 [8, 84]”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “The pronunciation of monophonic characters can be easily determined by a pronunciation dictionary, while G2P of polyphonic characters is highly context sensitive [262].”