Unsupervised clustering of emotion and voice styles for expressive TTS

By Florian Eyben, Sabine Buchholz, Norbert Braunschweiler, Javier Latorre, Vincent Wan, Mark John Francis Gales, K. Knill. 2012

73 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hung-yi Lee
2. Xu Tan
3. Yu Zhang
4. Tao Qin
5. Tie-Yan Liu
6. Ye Jia
7. Frank K. Soong
8. Ron J. Weiss

In Signal Processing

1. Hung-yi Lee
2. Yu Zhang
3. Ye Jia
4. Ron J. Weiss
5. Haizhou Li
6. Xu Tan
7. R. Skerry-Ryan
8. Frank K. Soong

Citations in the corpus

• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron (2018, methodology): “For example, (Eyben et al., 2012) proposes a system that first clusters the training set, and then performs HMM-based cluster-adaptive training.”
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis (2018, background): “Cluster-based modeling (Eyben et al., 2012; Jauk, 2017) is related to our work.”
• Improving Prosody Modelling with Cross-Utterance Bert Embeddings for End-to-End Speech Synthesis (2020, methodology): “Regarding the statistical parametric speech synthesis (SPSS) approach [6], styles are either pre-defined or automatically discovered, and can be modelled by individual transforms for hidden Markov models (HMMs) [7, 8].”
• Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder (2018, methodology): “[2, 18] pointed out that UESS can be divided into two parts: predicting expressive information from text; and synthesizing the speech with a particular expression.”
• Improving Unsupervised Style Transfer in end-to-end Speech Synthesis with end-to-end Speech Recognition (2018, methodology): “Cluster-based method [17, 18] uses some features, such as i-vector [19, 20], to cluster the training data, and trains a TTS model for each partition, but the clustering procedure and TTS are considered separately.”
• Uncovering Latent Style Factors for Expressive Speech Synthesis (2017, methodology): “For example, [7] proposes a system that first clusters the training set, and then performs HMM-based cluster-adaptive training.”
• Expressive TTS Training With Frame and Style Reconstruction Loss (2020, methodology): “for expressive TTS [33], [68], [86]–[88].”
• Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features (2019, methodology): “[8] proposed an approach involving unsupervised clustering of acoustic features and using cluster assignments as context for the generation process of average-expression speech synthesis.”
• Unsupervised Learning for Expressive Speech Synthesis (2017, background): “In [32] the authors use a set of prosodic parameters combined with some spectral ones; [11] use prosodic features, i.”