Speech Enhancement of Noisy and Reverberant Speech for Text-to-Speech

By Cassia Valentini-Botinhao, Junichi Yamagishi. 2018

36 citations recorded by the source corpus.

Text-to-speech voices created from noisy and reverberant recordings are of lower quality. A simple way to improve this is to increase the quality of the recordings prior to text-to-speech training with speech enhancement methods such as noise suppression and dereverberation. In this paper, we opted for this approach and to perform the enhancement, we used a recurrent neural network. The network is trained with parallel data of clean and lower quality recordings of speech. The lower quality data was artificially created by adding recordings of environmental noise to studio-quality recordings of speech and by convolving room impulse responses with these clean recordings. We trained separate networks with noise-only, reverberation-only, and both reverberation and additive noise data. The quality of voices trained with lower quality data that has been enhanced using these networks was significantly higher in all cases. For the noise-only case, the enhanced synthetic voice ranked as high as the voice trained with clean data. For the most realistic and challenging scenario, when both noise and reverberation were present, the improvements were more modest, but still significant.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Yi Ren
6. Sheng Zhao
7. Jinglin Liu
8. Zhou Zhao

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Yi Ren
6. Jinglin Liu
7. Sheng Zhao
8. Zhaoxi Mu

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “Some techniques such as speech enhancement [356], denoising [427], and disentangling [377, 117] can be leveraged to improve the quality of the speech data mined in the wild.”
• Speech Enhancement for Noise-Robust Speech Synthesis Using Wasserstein GAN (2019, methodology): “[8] proposed recently an SE technique for noise robust speech synthesis based on recurrent networks.”
• Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training (2020, methodology): “In order to build a robust TTS system, there are several attempts to conduct speech synthesis with noisy data [10, 11, 12].”
• Denoispeech: Denoising Text to Speech with Frame-Level Noise Modeling (2020, methodology): “A straight-forward way [1, 2, 3, 4] is to first de-noise the noisy speech with a pre-trained speech enhancement model [5], and then use the enhanced speech to train the TTS model.”
• Review of end-to-end speech synthesis technology based on deep learning (2021, background): “Valentini-Botinhao C, Yamagishi J (2018) Speech enhancement of noisy and reverberant speech for text-to-speech.”