Improving FFTNet Vocoder with Noise Shaping and Subband Approaches

By Takuma Okamoto, Tomoki Toda, Yoshinori Shiga, Hisashi Kawai. 2018

17 citations recorded by the source corpus.

Although FFTNet neural vocoders can synthesize speech waveforms in real time, the synthesized speech quality is worse than that of WaveNet vocoders. To improve the synthesized speech quality of FFTNet while ensuring real-time synthesis, residual connections are introduced to enhance the prediction accuracy. Additionally, time-invariant noise shaping and subband approaches, which significantly improve the synthesized speech quality of WaveNet vocoders, are applied. A subband FFTNet vocoder with multiband input is also proposed to directly compensate the phase shift between subbands. The proposed approaches are evaluated through experiments using a Japanese male corpus with a sampling frequency of 16 kHz. The results are compared with those synthesized by the STRAIGHT vocoder without mel-cepstral compression and those from conventional FFTNet and WaveNet vocoders. The proposed approaches are shown to successfully improve the synthesized speech quality of the FFTNet vocoder. In particular, the use of noise shaping enables FFTNet to significantly outperform the STRAIGHT vocoder.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Frank K. Soong
2. Dong Yu
3. Lei He
4. Heng Lu
5. Hisashi Kawai
6. Jinyu Li
7. Chao Weng
8. Junichi Yamagishi

In Computer Vision and Pattern Recognition

1. Meng Yu
2. Dong Yu
3. Dan Su
4. Chao Weng
5. Kun Xu
6. Shan Liu
7. Frank K. Soong
8. Tomoki Toda

In Signal Processing

1. Frank K. Soong
2. Dong Yu
3. Hisashi Kawai
4. Jinyu Li
5. Heng Lu
6. Takuma Okamoto
7. Lei He
8. Tomoki Toda

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, neutral): “[245] further enhance FFTNet with noise shaping and subband techniques, improving the voice quality while keeping small model size.”
• Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems (2019, methodology): “The first is AR models with simple structures, such as FFTNet [43, 44], WaveRNN [45], and LPCNet [46].”
• Triple M: A Practical Text-to-Speech Synthesis System with Multi-Guidance Attention and Multi-Band Multi-Time LPCNet (2021, methodology): “Multi-band processing is initially used for parallel acceleration of vocoder [16, 17].”
• FeatherWave: An efficient high-fidelity neural vocoder with multi-band linear prediction (2020, methodology): “S D ] 12 M a y 2020 with the technique of subband [12, 13] to improve generation speed.”
• An Efficient Subband Linear Prediction for LPCNet-Based Neural Synthesis (2020, background): “Some related works have investigated the neural vocoders in the subband domain, such as subband WaveNet[16, 17, 18], subband FFTNet[19], subband WaveRNN[20], FeatherWave[21], and multiband MelGan[22].”
• DurIAN: Duration Informed Attention Network For Multimodal Synthesis (2019, background): “While several studies [24, 25] have explored the usage of multi-band strategy for neural vocoder, these studies aim at modeling audio with higher sampling rate by training separate neural vocoder models for each subband.”
• DurIAN: Duration Informed Attention Network for Speech Synthesis (2020, methodology): “Existing approaches [23, 24] utilizing such charactertistics of multiband processing is to model each subband with separatly trained neural vocoder which will then be inferenced in parallel using mulitple CPUs or GPUs.”