Emotional End-to-End Neural Speech Synthesizer

By Younggun Lee, Azam Rabiee, Soo-Young Lee. 2017

123 citations recorded by the source corpus.

In this paper, we introduce an emotional speech synthesizer based on the recent end-to-end neural model, named Tacotron. Despite its benefits, we found that the original Tacotron suffers from the exposure bias problem and irregularity of the attention alignment. Later, we address the problem by utilization of context vector and residual connection at recurrent neural networks (RNNs). Our experiments showed that the model could successfully train and generate speech for given emotion labels.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lei Xie
2. Shan Yang
3. Yinjiao Lei
4. Taesu Kim
5. Slava Shechtman
6. Kyungguen Byun
7. Younggun Lee
8. Soo-Young Lee

In Signal Processing

1. Lei Xie
2. Shan Yang
3. Yinjiao Lei
4. Taesu Kim
5. Panagiotis Tzirakis
6. Kyungguen Byun
7. Soo-Young Lee
8. Younggun Lee

Citations in the corpus

• Fine-Grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis (2020, methodology): “The most straightforward way to conduct emotional speech synthesis is using explicit annotations or labels as global render to model expressions [19, 20].”
• An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era (2022, methodology): “Nowadays, with the advances seen in neural vocoders, it is typical to use those as the last step of the synthesis process; accordingly, EVC pipelines now concentrate more on modulating spectral features [50], [109], [112], [113], [120], [140].”
• Text aware Emotional Text-to-speech with BERT (2022, methodology): “[4] proposes to use the one-hot representation of the emotion label as the emotion embedding during the training.”
• Emotional Speech Synthesis with Rich and Granularized Control (2019, methodology): “, Tacotron), the expressiveness of synthesized speech is controlled by conditioning additional embedding vectors that implicitly provide prosodyrelated latent features typically generated by another neural network [8, 9, 10, 11, 12, 13].”
• Implementing Prosodic Phrasing in Chinese End-to-end Speech Synthesis (2019, unknown): “In a different study, an emotional end-to-end system was built by utilizing context vector and residual connection[17].”
• Controlling Emotion Strength with Relative Attribute for End-to-End Speech Synthesis (2019, unknown): “[26] have studied emotional TTS under E2E framework by simply adding emotion labels to Tacotron.”
• Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder (2018, background): “Many studies have reported that such labels are helpful for modeling complex audio data [5, 6, 4, 7].”
• Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis (2022, background): “cn/opencpop/ 16], emotional speech [17, 18, 19, 20, 21, 22], and also singing voice [23, 24].”
• Self-supervised Context-aware Style Representation for Expressive Speech Synthesis (2022, unknown): “In real applications, the supervised learning is more widely adopted by leveraging explicit labels as auxiliary information to guide multi-style TTS [9, 10].”
• Voice Imitating Text-to-Speech Neural Networks (2018, methodology): “Another study reported a TTS system that can generate voice containing emotions (Lee et al., 2017).”
• Controllable Emotion Transfer For End-to-End Speech Synthesis (2020, unknown): “A straight-forward way to conduct emotional speech synthesis is to use categorized emotional data to train a model [20].”
• Sequence to Sequence Neural Speech Synthesis with Prosody Modification Capabilities (2019, background): “(Semi-)Supervised approaches, that exploit prosodic and/or speaking style labeling (partially or fully generated by human subjects) [7].”