Adapting and controlling DNN-based speech synthesis using input codes

By Hieu-Thi Luong, Shinji Takaki, Gustav Eje Henter, Junichi Yamagishi. 2017

77 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis (2018, background): “However, most existing models require explicit labels, such as emotion or speaker codes (Luong et al., 2017).”
Uncovering Latent Style Factors for Expressive Speech Synthesis (2017, methodology): “This argument applies similarly to other conditioning attributes, such as emotion [9] and prominence [10].”
Linear Networks Based Speaker Adaptation for Speech Synthesis (2018, background): “Also, in some multi-speaker speech synthesis systems, i-vector and speaker code representing speaker [15, 16] are combined with linguistic features as input features for the neural network based acoustic model.”
Controlling Emotion Strength with Relative Attribute for End-to-End Speech Synthesis (2019, unknown): “Previous systems either simply render an averaged emotional output [5,6] or learn a prosodic space from ∗ Corresponding author.”
Expressive TTS Training With Frame and Style Reconstruction Loss (2020, unknown): “output in different ways [62].”
Model architectures to extrapolate emotional expressions in DNN-based text-to-speech (2021, methodology): “A key idea is to explicitly control the speaker factor and the emotional factor, motivated by the success in the multi-speaker model [6, 7, 8, 9, 10] and multi-emotional model [11, 12, 13].”
Improving Unsupervised Style Transfer in end-to-end Speech Synthesis with end-to-end Speech Recognition (2018, background): “Recently, some work try to improve multi-speaker TTS with speaker verification [12, 13].”
An investigation of multi-speaker training for wavenet vocoder (2017, background): “The main advantage of this approach is its high controllability, since it allows us to flexibly change the voice characteristics of synthesized speech by manipulating acoustic features [11, 12].”
NAUTILUS: A Versatile Voice Cloning System (2020, methodology): “As PPG is a stand-in for text, the adaptation techniques used for TTS such as adapting an average [37] or a multi-speaker [38] acoustic model”
Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder (2018, background): “Many studies have reported that such labels are helpful for modeling complex audio data [5, 6, 4, 7].”
Wasserstein GAN and Waveform Loss-Based Acoustic Model Training for Multi-Speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder (2018, background): “[21] proposed estimating code vectors for new speakers via backpropagation and experimented with manually manipulating”
Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis (2018, methodology): “2) We introduce and detail various theoretical connections between the techniques in [7], [10]–[12] and encoder-decoder models, particularly VQ-VAEs.”
Speaker Adaptation of a Multilingual Acoustic Model for Cross-Language Synthesis (2020, background): “In contrast, the speech synthesis techniques based on multi-layer feed-forward neural networks can be easily adapted using a small amount of data while maintaining naturalness as well as similarity to the target speaker [14].”