Neural Source-filter-based Waveform Model for Statistical Parametric Speech Synthesis

By Xin Eric Wang, Shinji Takaki, Junichi Yamagishi. 2018

132 citations recorded by the source corpus.

Neural waveform models such as the WaveNet are used in many recent text-to-speech systems, but the original WaveNet is quite slow in waveform generation because of its autoregressive (AR) structure. Although faster non-AR models were recently reported, they may be prohibitively complicated due to the use of a distilling training method and the blend of other disparate training criteria. This study proposes a non-AR neural source-filter waveform model that can be directly trained using spectrum-based training criteria and the stochastic gradient descent method. Given the input acoustic features, the proposed model first uses a source module to generate a sine-based excitation signal and then uses a filter module to transform the excitation signal into the output speech waveform. Our experiments demonstrated that the proposed model generated waveforms at least 100 times faster than the AR WaveNet and the quality of its synthetic speech is close to that of speech generated by the AR WaveNet. Ablation test results showed that both the sine-wave excitation signal and the spectrum-based training criteria were essential to the performance of the proposed model.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, background): “, 2020), and neural signal processing models (Wang et al., 2019; Engel et al., 2020; Ai & Ling, 2020).”
A Survey on Neural Speech Synthesis (2021, background): “Other Vocoders Some works leverage neural-based source-filter model for waveform generation [375, 374, 371, 207, 146, 145, 75, 305, 407], aiming to achieve high voice quality while maintaining controllable speech generation.”
A Spectral Energy Distance for Parallel Speech Synthesis (2020, neutral): “Our proposed spectral energy distance is closely related to other recent work in audio synthesis [37, 38, 9, 6], in that it is based on calculating distances between spectrograms, but we believe it is the first to include a repulsive term between generated samples, and thus the first proper scoring ”
A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet (2019, methodology): “Other models use linear prediction to remove the burden of spectral envelope modeling from the neural synthesis network [12, 13, 14].”
End-to-End Adversarial Text-to-Speech (2020, methodology): “Spectrogram prediction losses have been used extensively for feed-forward audio prediction models (Yamamoto et al., 2019; 2020; Yang et al., 2020; Arık et al., 2018; Engel et al., 2020; Wang et al., 2019).”
Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations (2021, methodology): “By modeling source and filter components with deep networks, it has been used for applications such as vocoder [49, 23, 45] and acoustic feature generation [25].”
Neural Homomorphic Vocoder (2020, methodology): “Multi-resolution STFT loss is tolerant of phase mismatch in signals [10, 13, 15, 17].”
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection (2019, methodology): “The 11 unknown systems comprise 2 VC, 6 TTS and 3 hybrid TTS-VC systems and were implemented with various waveform generation methods including classical vocoding, GriffinLim [9], generative adversarial networks [10], neural waveform models [8, 11], waveform concatenation, waveform filtering [12], s”
Knowledge-and-Data-Driven Amplitude Spectrum Prediction for Hierarchical Neural Vocoders (2020, methodology): “The neural waveform generator is built by adapting the NSF vocoder [23] from three aspects, 1) using LAS as the input, 2) pre-calculating the initial phase of the sine-based excitation signal for each voiced segment at the training stage and 3) adopting a combined loss function including MSE on ampl”
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech (2019, methodology): “However, A08 uses a neural-source-filter waveform model [29], which is much faster than WaveNet.”
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning (2020, unknown): “a neural source-filter (NSF) waveform modeling framework is proposed [106], [107].”
Quasi-Periodic Parallel WaveGAN Vocoder: A Non-autoregressive Pitch-dependent Dilated Convolution Model for Parametric Speech Generation (2020, background): “Since the pitch controllability is an essential feature for a vocoder, NN-based SS models with carefully designed periodic and aperiodic inputs [24–26] greatly improve the pitch modeling capability.”
Chunked Autoregressive GAN for Conditional Waveform Synthesis (2021, methodology): “, 2020), the Neural Source-Filter (NSF) model (Wang et al., 2019), and the GAN-based HiFi-GAN (Kong et al.”
Neural Harmonic-plus-Noise Waveform Model with Trainable Maximum Voice Frequency for Text-to-Speech Synthesis (2019, methodology): “In our previous work, we proposed NSF models [14] to convert c1:B into ô1:T .”
Controllable Neural Prosody Synthesis (2020, background): “Existing vocoders allow either perceptually high-quality synthesis [3, 13, 14] or a high degree of control over prosody [15].”
Unsupervised Cross-Domain Singing Voice Conversion (2020, methodology): “To mitigate this, we include a spectral amplitude distance loss [12, 39] in multiple FFT resolutions [3, 13, 14].”