Model architectures to extrapolate emotional expressions in DNN-based text-to-speech

By Katsuki Inoue, Sunao Hara, Masanobu Abe, Nobukatsu Hojo, Yusuke Ijima. 2021

16 citations recorded by the source corpus.

Abstract In this paper, we propose architectures that make it possible to extrapolate emotional expressions in deep neural network (DNN)-based text-to-speech (TTS). In this study, the meaning of extrapolate emotional expressions is to borrow emotional expressions from someone else, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power, and several types of DNN-based TTS have demonstrated satisfactory performances in the expression of the diversity of human speech, to construct DNN-based TTS with emotional expressions, it is necessary and troublesome to collect emotional speech uttered by target speakers. To solve the problem, we propose architectures to separately train speaker factor and emotional factor, and synthesize speech with any combined quality of speakers and emotions. The architectures are parallel model (PM), serial model (SM), auxiliary input model (AIM), and those hybrid models. These models trained by using emotional speech uttered by few speakers and neutral speech uttered by many speakers. Objective evaluations demonstrate that the performances in the open-emotion test are insufficient by making a comparison with those in the closed-emotion test, because each speaker has his/her own manner of expressing emotion. However, subjective evaluation results indicate that the proposed models could convey emotional information to some extent. Notably, the PM can correctly convey sad and joyful emotions at a rate of > 60 %.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Yi Ren
7. Zhou Zhao
8. Lei He

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “In this subsection, we review techniques on disentangling [218, 117, 275], controlling [353, 180, 240, 13, 267, 343, 192], and transferring [150, 131, 393, 6] variation information, as shown in Table 15.”