Speech Super-Resolution Using Parallel WaveNet

By Mu Wang, Zhiyong Wu, Shiyin Kang, Xixin Wu, Jia Jia, Dan Su, Dong Yu, Helen Mei-Ling Meng. 2018

21 citations recorded by the source corpus.

Audio super-resolution is the task to increase the sampling rate of a given low-resolution (i.e. low sampling rate) audio. One of the most popular approaches for audio super-resolution is to minimize the squared Euclidean distance between the reconstructed signal and the high sampling rate signal in a point-wise manner. However, such approach has intrinsic limitations, such as the regression to mean problem. In this work, we introduce a novel auto-regressive method for the speech super-resolution task, which utilizes WaveNet to model the distribution of the target high-resolution signal conditioned on the log-scale mel-spectrogram of the low-resolution signal. As an auto-regressive neural network, WaveNet uses the negative log-likelihood as the objective function, which is much more suitable for highly stochastic process such as speech waveform, instead of the Euclidean distance. We also train a parallel WaveNet to speed up the generating process to real-time. In the experiments, we perform speech super-resolution by increasing the sampling rate from 4kHz to 16kHz on the VCTK corpus. The proposed method can achieve the improvement of ∼2 dB over the base-line deep residual convolutional neural network (CNN) under the Log-Spectral Distance (LSD) metric.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Real-Time Speech Frequency Bandwidth Extension (2020, methodology): “Inspired by the early success in image super-resolution [4], end-to-end audio-based solutions were proposed, based on wave-to-wave UNet [5], WaveNet [6, 7], hybrid time/frequency domain models [8].”
High Fidelity Speech Regeneration with Application to Speech Enhancement (2021, methodology): “The authors in [14] suggest to use a WaveNet model to directly output a high sampled speech signal while the authors in [15] suggest using GANs to estimate the mel-spectrogram and then apply a vocoder to generate enhanced waveform.”