EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System

By Hao Li, Yongguo Kang, Zhenyu Wang. 2018

22 citations recorded by the source corpus.

We present EMPHASIS, an emotional phoneme-based acoustic model for speech synthesis system. EMPHASIS includes a phoneme duration prediction model and an acoustic parameter prediction model. It uses a CBHG-based regression network to model the dependencies between linguistic features and acoustic features. We modify the input and output layer structures of the network to improve the performance. For the linguistic features, we apply a feature grouping strategy to enhance emotional and prosodic features. The acoustic parameters are designed to be suitable for the regression task and waveform reconstruction. EMPHASIS can synthesize speech in real-time and generate expressive interrogative and exclamatory speech with high audio quality. EMPHASIS is designed to be a multi-lingual model and can synthesize Mandarin-English speech for now. In the experiment of emotional speech synthesis, it achieves better subjective results than other real-time speech synthesis systems.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Sheng Zhao
7. Yi Ren
8. Helen Mei-Ling Meng

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Songxiang Liu
7. Jian Luan
8. Helen Mei-Ling Meng

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Acoustic Model Input→Output AR/NAR Modeling Structure HMM-based [409, 350] Ling→MCC+F0 / / HMM DNN-based [419] Ling→MCC+BAP+F0 NAR / DNN LSTM-based [76] Ling→LSP+F0 AR / RNN EMPHASIS [187] Ling→LinS+CAP+F0 AR / Hybrid ARST [369] Ph→LSP+BAP+F0 AR Seq2Seq RNN VoiceLoop [327] Ph→MGC+BAP+F0 AR / hybrid”
• HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis (2020, background): “2 For example, a 1 second audio waveform contains 48,000 sampling points when sampling rate is 48kHz. synthesis (Wu et al.; Li et al., 2018), to neural network based parametric synthesis (Arik et al., 2017), and to currently end-to-end neural models.”
• DeviceTTS: A Small-Footprint, Fast, Stable Network for On-Device Text-to-Speech (2020, unknown): “We use 67-dim WORLD features, comprising 60-dim mel-cepstral coefficients, 3-dim log F0 ( static + ∆ + ∆∆ ), 3-dim aperiodicity of center frequencies (CAP) [15] and 1-dim UV flag.”
• End-to-end Code-switched TTS with Mix of Monolingual Recordings (2019, background): “(ii) Multilingual TTS systems that use separated front-ends to process the text portions of different languages while sharing a unit-selection module [4] or acoustic model [5, 6].”
• Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS (2020, methodology): “Refer to [21], which uses both phoneme-related features and prosody-related features as their neural TTS input, the framework of our linguistic feature based model is shown in Fig 2.”