Statistical parametric speech synthesis using deep neural networks

By Heiga Zen, Andrew W. Senior, Mike Schuster. 2013

924 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Junichi Yamagishi
4. Tie-Yan Liu
5. Frank K. Soong
6. Dong Yu
7. Graham Neubig
8. Jinyu Li

In Speech & Audio

1. Xu Tan
2. Junichi Yamagishi
3. Lei He
4. Tao Qin
5. Sheng Zhao
6. Yu Zhang
7. Frank K. Soong
8. Tie-Yan Liu

In Signal Processing

1. Junichi Yamagishi
2. Xu Tan
3. Frank K. Soong
4. Tao Qin
5. Tie-Yan Liu
6. Jinyu Li
7. Dong Yu
8. Shinji Watanabe

Citations in the corpus

• Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions (2017, background): “Statistical parametric speech synthesis [4, 5, 6, 7], which directly generates smooth trajectories of speech features to be synthesized by a vocoder, followed, solving many of the issues that concatenative synthesis had with boundary artifacts.”
• Neural Speech Synthesis with Transformer Network (2018, methodology): “Parametric TTS systems (Tokuda et al. 2000; Zen, Tokuda, and Black 2009; Ze, Senior, and Schuster 2013; Tokuda et al. 2013) convert speech waves into spectrograms, and acoustic parameters, such as fundamental frequency and duration, are used to synthesize new audio results.”
• Deep Voice: Real-time Neural Text-to-Speech (2017, methodology): “WaveNet uses several features from a TTS system (Zen et al., 2013), that include values such as the number of syllables in a word, position of syllables in the phrase, position of the current frame in the phoneme, and dynamic features of the speech spectrum like spectral and excitation parameters, a”
• UniAudio: An Audio Foundation Model Toward Universal Audio Generation (2023, background): “…(Yu et al., 2017) or others (Gu et al., 2021; Shen et al., 2023); (3) their designs are inspired by and derived from linguistics and phonetics (Zen et al., 2013), signal processing (Griffin & Lim, 1984), auditory perception (Shadle & Damper, 2001) and machine learning (Wang et al., 2016)…”
• A Survey on Neural Speech Synthesis (2021, methodology): “From 2010s, neural network-based speech synthesis [419, 278, 76, 417, 369, 187, 248, 376] has gradually become the dominant methods and achieved much better voice quality.”
• Fine-Grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis (2020, methodology): “he phoneme-level speech fragments to compute the emotion strength as local emotion descriptors through the learned ranking function. The extracted phoneme-level descriptors are finally normalized into [0,1]. Table 1: MCD of different models for parallel transfer Method MCD (dB) GST 4.89 UET 5.16 prop”
• TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese (2020, methodology): “Finally, the acoustic model receives the phoneme representation sequence, the prosodic information about phoneme segments’ length, the F0 contour and computes the speech signal (Teixeira et al., 2003; Ze et al., 2013).”
• Fast, Compact, and High Quality LSTM-RNN Based Statistical Parametric Speech Synthesizers for Mobile Devices (2016, methodology): “desirable to reduce the amount of computations at the inference stage. In typical ANN-based SPSS, input linguistic features other than state- and frame-position features are constant within a phoneme [2]. Furthermore, speech is a rather stationary process at 5-ms frame shift and target acoustic fram”
• MusicLM: Generating Music From Text (2023, background): “Conditional neural audio generation covers a wide range of applications, ranging from text-to-speech (Zen et al., 2013; van den Oord et al., 2016) to lyrics-conditioned music generation (Dhariwal et al., 2020) and audio synthesis from MIDI sequences (Hawthorne et al., 2022b).”
• Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis (2019, background): “2956145 have been reported for the acoustic modeling part, for which various types of neural networks have been proposed that convert linguistic features into acoustic features such as the fundamental frequency (F0) [5], [6], cepstral or spectral features [7]–[9], and duration [10], [11].”
• Parallel Tacotron: Non-Autoregressive and Controllable TTS (2020, background): “They are analogous to conventional parametric synthesis models [25, 26], where phoneme durations are often modeled explicitly.”
• Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding (2016, methodology): “introduction of DNNs in the field of parametric speech synthesis as well, constantly outperforming HMM-based speech synthesis systems [21], [24], [25].”
• DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021 (2021, methodology): “Rich unit selection based concatenative speech synthesis [8] and statistical parametric speech synthesis like HMM/DNN/LSTM based models [9, 10, 11], have been the most popular methods in the past.”
• Listening while speaking: Speech chain by deep learning (2017, positive): “r the resurgence of deep learning, interest has also surfaced in the possibility of utilizing a neural approach for ASR and TTS systems. Many state-of-the-art performances in ASR [13, 14, 15] and TTS [16, 17, 18] tasks have been successfully constructed based on neural network frameworks. However, d”
• On the training of DNN-based average voice model for speech synthesis (2016, methodology): “Although the GMM-HMM framework can model the relationship between linguistic features and acoustic parameters, the decision tree based clustering cannot generalise well for unseen context and limits the naturalness of synthesised speech [3], [4].”