LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation

By Zhen Zeng, Jianzong Wang, Ning Cheng, Jing Xiao. 2021

18 citations recorded by the source corpus.

In this paper, we propose a novel conditional convolution network, named location-variable convolution, to model the dependencies of the waveform sequence. Different from the use of unified convolution kernels in WaveNet to capture the dependencies of arbitrary waveform, the location-variable convolution uses convolution kernels with different coefficients to perform convolution operations on different waveform intervals, where the coefficients of kernels is predicted according to conditioning acoustic features, such as Mel-spectrograms. Based on location-variable convolutions, we design LVCNet for waveform generation, and apply it in Parallel WaveGAN to design more efficient vocoder. Experiments on the LJSpeech dataset show that our proposed model achieves a four-fold increase in synthesis speed compared to the original Parallel WaveGAN without any degradation in sound quality, which verifies the effectiveness of location-variable convolutions.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Congyi Wang
7. Yanli Li
8. Yi Ren

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Frank K. Soong
5. Congyi Wang
6. Yanli Li
7. D. Lim
8. Jaesam Yoon

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Lightweight Model WaveRNN [147], LightSpeech [214] SqueezeWave [426], [148, 114, 123, 424]”
• UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation (2021, methodology): “The kernels of the LVC layers are predicted using a kernel predictor that takes the log-mel-spectrogram as the input.”
• Improve GAN-based Neural Vocoder using Truncated Pointwise Relativistic Least Square GAN (2022, methodology): “Most existing GAN-based vocoders([1, 2, 5, 13–15, 19, 20, 28, 31– 34, 37]) adopt the least square GAN[16](LSGAN) framework and multiple discriminators to improve their generation quality.”
• Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GAN (2021, methodology): “GAN-based methods([17],[18],[19],[20],[21],[22],[23]) are promising given some models can even synthesize waves in realtime on a single GPU and achieve a higher MOS at the same time, suited for actual industrial deployment.”