PHONEME-LEVEL BERT FOR ENHANCED PROSODY OF TEXT-TO-SPEECH WITH GRAPHEME PREDICTIONS
By Cong Han, Xilin Jiang, Nima Mesgarani. 2023
37 citations recorded by the source corpus.
Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns. However, these models are usually word-level or sup-phoneme-level and jointly trained with phonemes, making them inefficient for the downstream TTS task where only phonemes are needed. In this work, we propose a phoneme-level BERT (PL-BERT) with a pretext task of predicting the corresponding graphemes along with the regular masked phoneme predictions. Subjective evaluations show that our phoneme-level BERT encoder has significantly improved the mean opinion scores (MOS) of rated naturalness of synthesized speech compared with the state-of-the-art (SOTA) StyleTTS baseline on out-of-distribution (OOD) texts.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Cong Han
- Vinay S. Raghavan
- Nima Mesgarani
- Gavin Mischler
- Yi Luo
- Eliya Nachmani
- Zhuo Chen
- Jonathan Le Roux
Citations in the corpus
- StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models (2023, methodology): “VITS [3] demonstrates MOS comparable to human recordings on the LJSpeech and VCTK datasets, while PnG-BERT [4] obtains human-level results on a proprietary dataset.”