A Speech Parameter Generation Algorithm Considering Global Variance for HMM-Based Speech Synthesis

By Tomoki Toda, Keiichi Tokuda. 2007

505 citations recorded by the source corpus.

This paper describes a novel parameter generation algorithm for an HMM-based speech synthesis technique. The conventional algorithm generates a parameter trajectory of static features that maximizes the likelihood of a given HMM for the parameter sequence consisting of the static and dynamic features under an explicit constraint between those two features. The generated trajectory is often excessively smoothed due to the statistical processing. Using the over-smoothed speech parameters usually causes muffled sounds. In order to alleviate the over-smoothing effect, we propose a generation algorithm considering not only the HMM likelihood maximized in the conventional algorithm but also a likelihood for a global variance (GV) of the generated trajectory. The latter likelihood works as a penalty for the over-smoothing, i.e., a reduction of the GV of the generated trajectory. The result of a perceptual evaluation demonstrates that the proposed algorithm causes considerably large improvements in the naturalness of synthetic speech.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

WaveNet: A Generative Model for Raw Audio (2016, background): “…developing high-quality vocoders (Kawahara et al., 1999; Agiomyrgiannakis, 2015; Morise et al., 2016), improving the accuracy of generative models (Zen et al., 2007; 2013; Fan et al., 2014; Uria et al., 2015), and compensating the oversmoothing effect (Toda & Tokuda, 2007; Takamichi et al., 2016).”
A Survey on Neural Speech Synthesis (2021, background): “Modeling the one-to-many mapping under the regular L1 loss [83, 354] without enough input information will cause over-smoothing mel-spectrogram prediction [347, 328], e,g.”
Speech Enhancement of Noisy and Reverberant Speech for Text-to-Speech (2018, methodology): “In the work described in [6] authors train a feedforward neural network with noise-aware training and global variance estimation using more than 100 different noise conditions [10], both techniques seem to improve results.”
DNN-based stochastic postfilter for HMM-based speech synthesis (2014, methodology): “The over-smoothing can be partly alleviated by speech parameter generation considering global variance (GV) [3].”
Modeling Spectral Envelopes Using Restricted Boltzmann Machines and Deep Belief Networks for Statistical Parametric Speech Synthesis (2013, methodology): “As we discussed in Section I, there are also some other approaches that can significantly reduce the over-smoothing and improve the quality of the synthetic speech, such as the GV-based parameter generation [11] and the post-filtering techniques [6], [13].”
Connectionist multivariate density-estimation and its application to speech synthesis (2016, methodology): “To reduce this over-smoothing, postfiltering and generation techniques that take into account the variance of the acoustic trajectory are usually applied to modify the acoustics generated (Tomoki and Tokuda, 2007).”
A deep auto-encoder based low-dimensional feature extraction from FFT spectral envelopes for statistical parametric speech synthesis (2016, methodology): “In this paper, we didn’t apply spectral enhancement techniques such as global variance [28] or post-filtering [6] to all systems to reduce factors considered in the listening test.”
Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis (2015, methodology): “Significant efforts have been made to improve the acoustic models for SPSS, targeting the underlying model, the way the parameters are estimated during training, or the method for generating speech parameter trajectories when performing synthesis: minimum generation error training for HMM [4], globa”
Autoregressive Models for Statistical Parametric Speech Synthesis (2013, background): “1 is an autoregressive HMM with depth 0.”
Speech Enhancement for a Noise-Robust Text-to-Speech Synthesis System Using Deep Recurrent Neural Networks (2016, methodology): “A more extensive work on speech enhancement using DNNs is presented in [6] where authors use more than 100 noise types to train a feed-forward network using noise-aware training and global variance [9].”
Robust Speaker-Adaptive HMM-Based Text-to-Speech Synthesis (2009, methodology): “Finally, we explain the GV parameter generation algorithm [22] for the CSMAPLR adapted model.”
Text-to-Speech Synthesis (2009, methodology): “Durations within phones are modelled by hidden semi-Markov models, an d global variance technique [451] is used to compensate for the over smoothing effect tha t HMM synthesis can create.”
Utterance Selection Techniques for TTS Systems Using Found Speech (2016, methodology): “This was another global spectral measure we tried since it improves results when used as a post-ﬁlter [18].”
Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech (2016, unknown): “A more extensive work on speech enhancement using DNNs is presented in [6] where authors use more than 100 noise types to train a feed-forward network using noise-aware training and global variance [10].”
Model architectures to extrapolate emotional expressions in DNN-based text-to-speech (2021, methodology): “The variance of these features was expanded to the global variance (GV) [27], extracted from the target neutral speech, by using variance scaling [28]. between closed-and open-emotion tests is smaller than the difference between Resyns and closed-emotion tests.”
A Deep Generative Architecture for Postfiltering in Statistical Parametric Speech Synthesis (2015, methodology): “Another method frequently used for improving the quality of synthetic speech is a parameter generation algorithm that considers GV [15].”
Improving Unsupervised Style Transfer in end-to-end Speech Synthesis with end-to-end Speech Recognition (2018, methodology): “This property can be verified by calculating the Global Variance (GV) [39] over the spectrogram.”
Spoofing and countermeasures for speaker verification: A survey (2015, methodology): “This method works well in detecting HMM-based synthetic speech without global variance compensation (Tomoki and Tokuda, 2007).”
Edinburgh Research Explorer Modelling acoustic feature dependencies with artificial neural networks: Trajectory-RNADE (unknown): “To reduce this over-smoothing, postﬁltering and generation techniques that take into account the variance of the acoustic trajectory are commonly used [3].”