Noise Level Limited Sub-Modeling for Diffusion Probabilistic Vocoders

By Takuma Okamoto, Tomoki Toda, Yoshinori Shiga, Hisashi Kawai. 2021

11 citations recorded by the source corpus.

Although diffusion probabilistic vocoders WaveGrad and DiffWave can realize real-time high-fidelity speech synthesis with a simple loss function in training, all noise components with over the full range of noise levels are predicted by one model in all iterations. This paper proposes a simple but effective noise level-limited sub-modeling framework for diffusion probabilistic vocoders Sub-WaveGrad and Sub-DiffWave. In the proposed method, DiffWave conditioned on a continuous noise level like WaveGrad, and spectral enhancement post-filtering are also provided. The proposed Sub-WaveGrad and Sub-DiffWave models are realized using 10 sub-models. These models are separately trained with different noise level limits, and only necessary sub-models are used according to the noise schedule during inference. The results of experiments using a Japanese female speech corpus indicate that both the proposed Sub-WaveGrad and Sub-DiffWave outperform vanilla WaveGrad and DiffWave in terms of the model accuracy and synthesis quality while retaining the inference speed.

Subfields: Artificial Intelligence, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wei Ping
2. Heiga Zen
3. Nanxin Chen
4. Michiel Bacchiani
5. Zhifeng Kong
6. Kohei Yatabe
7. Yuma Koizumi
8. Junichi Yamagishi

In Generative Models

1. Wei Ping
2. Zhifeng Kong
3. Nanxin Chen
4. Heiga Zen
5. Michiel Bacchiani
6. Kohei Yatabe
7. Yuma Koizumi
8. Jonathan Ho

In Speech & Audio

1. Wei Ping
2. Michiel Bacchiani
3. Heiga Zen
4. Kohei Yatabe
5. Yuma Koizumi
6. Nanxin Chen
7. Zhifeng Kong
8. Junichi Yamagishi

In Signal Processing

1. Wei Ping
2. Heiga Zen
3. Nanxin Chen
4. Zhifeng Kong
5. Michiel Bacchiani
6. Kohei Yatabe
7. Yuma Koizumi
8. Junichi Yamagishi

Citations in the corpus

• On Fast Sampling of Diffusion Probabilistic Models (2021, methodology): “They retrain the network conditioned on continuous noise levels instead of discrete diffusion steps (Song and Ermon, 2019; Chen et al., 2020; Okamoto et al., 2021; San-Roman et al., 2021).”
• Wavefit: an Iterative and Non-Autoregressive Neural Vocoder Based on Fixed-Point Iteration (2022, unknown): “To reduce the number of iterations while maintaining the quality, existing studies of DDPMs have investigated the inference noise schedule [44], the use of adaptive prior [45, 46], the network architecture [47, 48], and/or the training strategy [49].”
• SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping (2022, background): “To reduce the number of iterations while maintaining the quality, the conventional studies have proposed a proper inference noise schedule [25, 26] and/or network architecture [27, 28].”